CN117768765A

CN117768765A - Abnormal image identification method and device, electronic equipment and storage medium

Info

Publication number: CN117768765A
Application number: CN202211139453.3A
Authority: CN
Inventors: 张培磊; 李熊; 刘洋; 杨宇航
Original assignee: Beijing Sankuai Online Technology Co Ltd
Current assignee: Beijing Sankuai Online Technology Co Ltd
Priority date: 2022-09-19
Filing date: 2022-09-19
Publication date: 2024-03-26

Abstract

The embodiment of the application provides a method, a device, electronic equipment and a storage medium for identifying an abnormal image, which can accurately identify the abnormal image from an image sensor, so that the flight safety of an aircraft is ensured. The method for identifying the abnormal image comprises the following steps: receiving a plurality of Raw images acquired by an image sensor; determining a Raw image which meets preset conditions in the plurality of Raw images as an abnormal Raw image; and outputting alarm information, wherein the alarm information is used for prompting the abnormal type of the abnormal Raw image.

Description

Abnormal image identification method and device, electronic equipment and storage medium

[ field of technology ]

The embodiment of the application relates to the technical field of aircrafts, in particular to a method and a device for identifying abnormal images, electronic equipment and a storage medium.

[ background Art ]

Currently, during the flight of an aircraft, an image sensor may be abnormal during the process of acquiring an image, or may be abnormal during the process of transmitting an image to an image processor, so that an image of Raw sent to the image processor is abnormal, and the subsequent flight process of guiding the aircraft based on the abnormal image of Raw may affect the flight safety of the aircraft. Therefore, how to identify an abnormal image from an image sensor is a problem to be solved.

[ invention ]

The embodiment of the application provides a method, a device, electronic equipment and a storage medium for identifying an abnormal image, which can accurately identify the abnormal image from an image sensor, so that the flight safety of an aircraft is ensured.

In a first aspect, an embodiment of the present application provides a method for identifying an abnormal image, where the method includes:

receiving a plurality of Raw images acquired by an image sensor;

determining the Raw images meeting the preset condition in the plurality of Raw images as abnormal Raw images, wherein the preset condition is that the number of the Raw images received at adjacent actual receiving moments is lower than a first set threshold value, and/or the pixel values of adjacent pixels exceeding a second set threshold value in the same Raw image are the same, and/or the number of the actual pixels included in the same Raw image is lower than the number of target pixels, and/or the time difference between the expected receiving moment and the actual receiving moment of any Raw image is greater than a third set threshold value, and/or the number of pixels lacking color channels in the same Raw image is greater than a fourth set threshold value;

and outputting alarm information, wherein the alarm information is used for prompting the abnormal type of the abnormal Raw image.

In the embodiment of the present application, for a plurality of Raw images generated by the image sensor, if the number of Raw images received at adjacent expected times is insufficient, it may be considered that image loss or inter-frame freezing occurs; if the pixel values of a plurality of adjacent pixel points in the same Raw image are the same, the intra-frame freezing abnormality can be considered to occur; if the number of the actual pixel points in the same Raw image is lower than the number of the target pixel points, the pixel points are considered to be lost abnormally; if the expected receiving time and the actual receiving time of the same Raw image have larger differences, the delay abnormality can be considered to occur; if the number of pixel points of the color channel which is absent in the same Raw image is large, channel missing abnormality can occur. If the at least one abnormal image is identified from the plurality of Raw images, alarm information can be output, and related staff can be informed of the abnormal type of the abnormal image, so that the related staff can take corresponding measures in time, and the flight safety of the aircraft is improved.

Optionally, the determining the Raw image meeting the preset condition in the plurality of Raw images as the abnormal Raw image includes:

acquiring image sequence numbers of the Raw images received at the adjacent actual receiving moments, wherein the image sequence numbers are used for representing the generation sequence of the Raw images;

and if the actual change amplitude of the image sequence number in the adjacent actual receiving time is smaller than the first set threshold, taking the Raw image received in the adjacent expected receiving time as the abnormal Raw image, wherein the abnormal type of the abnormal Raw image is inter-frame freezing.

In this embodiment of the present application, in a process that the image sensor sequentially generates the Raw images, it may be considered that each Raw image is given an image sequence number representing a generation sequence thereof, and if the image sensor suddenly fails in a gap between adjacent Raw images, then the image sensor may not generate a new Raw image in a subsequent time until the image sensor returns to normal again. Therefore, if the actual change range of the image sequence numbers of the Raw images received at the adjacent actual reception times is smaller than the first set threshold, it is considered that an abnormal Raw image exists and the abnormal type of the abnormal Raw image is inter-frame freezing.

and if the number of times that the same pixel value continuously appears in the pixel points at the adjacent positions in one row of the row images exceeds the second set threshold, taking the row images as the abnormal row images, wherein the abnormal type of the abnormal row images is intra-frame freezing.

In this embodiment of the present application, if the image sensor fails during the process of generating the Raw image, the pixels formed subsequently are in a repeated state from the time of failure, that is, the pixel sequence numbers of the pixels formed subsequently are the same. Therefore, after the processor receives the Raw image, if it is determined that the number of times that the same pixel value continuously appears in the pixel points at adjacent positions is greater, it is considered that an abnormal Raw image exists, and the abnormal type of the abnormal Raw image is intra-frame freezing.

and if the image sequence numbers jump in the adjacent actual receiving moments, taking the Raw images received by the adjacent actual receiving moments as the abnormal Raw images, wherein the abnormal types of the abnormal Raw images are inter-frame loss.

In the embodiment of the application, in the process that the image sensor sequentially generates a plurality of Raw images, each Raw image can be considered to be given an image sequence number representing the generation sequence of the Raw image. Therefore, if a jump occurs between the respective image sequence numbers of the Raw images received at adjacent actual receiving moments, it can be considered that there is an abnormal Raw image, and the abnormal type of the abnormal Raw image is inter-frame loss, that is, a part of the Raw image is lost during transmission.

Optionally, the determining the Raw image meeting the preset condition in the plurality of Raw images as the abnormal Raw image of the target abnormal type includes:

acquiring a pixel point set sequence number of a pixel point set in the Raw image, wherein the pixel point set sequence number is used for representing the position of the pixel point set in the Raw image;

and if the pixel point set sequence numbers of the adjacent pixel point sets in the Raw image jump, taking the Raw image as the abnormal Raw image, wherein the abnormal type of the abnormal Raw image is intra-frame loss.

In the embodiment of the application, in the process of generating any Raw image by the image sensor, each pixel point set in the Raw image can be considered to be endowed with a pixel point set sequence number representing the position of the pixel point set. Therefore, if a jump occurs between the pixel point set numbers between the adjacent pixel point sets in a certain Raw image, it can be considered that an abnormal Raw image exists, and the abnormal type of the abnormal Raw image is intra-frame loss, that is, a part of pixel point sets in the Raw image are lost in the transmission process.

determining expected receiving time of an ith Raw chart in the plurality of Raw images based on time T1 when a configuration instruction is sent to the image sensor end and fixed delay time, wherein the configuration instruction is used for configuring the frame rate and the exposure time of the image sensor, and the fixed delay time is positively correlated with i;

obtaining the actual receiving time of the i-piece Raw image;

if the time difference between the actual receiving time and the expected receiving time is greater than the third set threshold, determining the ith Raw image as the abnormal Raw image, wherein the abnormal type of the abnormal Raw image is a timeout type;

in the embodiment of the present application, in the process of generating any Raw image by the image sensor, the Raw image may be considered to have a corresponding expected receiving time. If the actual receiving time of the Raw image is more than the expected receiving time, the abnormal Raw image exists in the transmission process, and the abnormal type of the abnormal Raw image is a timeout type.

acquiring an actual RGB value corresponding to each pixel point of the Raw graph;

and if the number of the pixels with different actual RGB values from the corresponding expected RGB values in the Raw image is larger than the fourth set threshold, determining the Raw image as the abnormal Raw image, wherein the abnormal type of the abnormal Raw image is color channel missing.

In this embodiment of the present application, in the process of generating any Raw image by the image sensor, it may be considered that each pixel point included in the Raw image has a corresponding RGB value. After the Raw image is received, if the RGB values of the pixel points at more positions in the Raw image are determined to be inconsistent with the expectations, the received Raw image can be considered as an abnormal Raw image, and the abnormal type of the abnormal Raw image is color channel missing.

Optionally, the image data of the Raw image includes information of an image serial number of the Raw image and/or information of a pixel point set serial number.

In the embodiment of the application, the image data of the Raw image includes the image sequence number of the Raw image and/or the pixel point set sequence number, so that the image sequence number corresponding to the Raw image and the internal pixel point set sequence number can be conveniently acquired while the Raw image is acquired.

In a second aspect, an embodiment of the present application provides an apparatus for identifying an abnormal image, including:

a receiving unit for receiving a plurality of Raw images generated by the image sensor;

a determining unit, configured to determine, as an abnormal Raw image, raw images that meet a preset condition in the plurality of Raw images, where the preset condition is that a number of Raw images received at adjacent actual receiving moments is lower than a first set threshold, and/or pixel values of adjacent pixels exceeding a second set threshold in the same Raw image are the same, and/or a number of actual pixels included in the same Raw image is lower than a number of target pixels, and/or a time difference between an expected receiving moment and an actual receiving moment of any Raw image is greater than a third set threshold, and/or a number of pixels lacking a color channel in the same Raw image is greater than a fourth set threshold;

the output unit is used for outputting alarm information, and the alarm information is used for prompting the abnormal type of the abnormal Raw image.

Optionally, the determining unit is specifically configured to:

and if the image sequence numbers jump in the adjacent actual receiving moments, taking the Raw images received by the adjacent actual receiving moments as the abnormal Raw images, wherein the abnormal type of the abnormal Raw images is inter-frame loss.

Optionally, the determining unit is specifically configured to:

obtaining the actual receiving moment of the ith Raw image;

and if the time difference between the actual receiving time and the expected receiving time is greater than the third set threshold, determining the ith Raw image as the abnormal Raw image, wherein the abnormal type of the abnormal Raw image is a timeout type.

Optionally, the determining unit is specifically configured to:

acquiring an actual RGB value corresponding to each pixel point of the Raw image;

and if the number of the pixels with different actual RGB values from the expected RGB values in the Raw image is larger than the fourth set threshold, determining the Raw image as the abnormal Raw image, wherein the abnormal type of the abnormal Raw image is color channel missing.

In a third aspect, embodiments of the present application provide an electronic device comprising a processor and a memory, the processor being configured to implement the steps of the method according to any of the embodiments of the first aspect when executing a computer program stored in the memory.

In a fourth aspect, embodiments of the present application provide a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements the steps of the method according to any of the embodiments of the first aspect.

It should be understood that the second to fourth aspects of the embodiments of the present application are consistent with the technical solutions of the first aspect of the embodiments of the present application, and the beneficial effects obtained by each aspect and the corresponding possible implementation manner are similar, and are not repeated.

[ description of the drawings ]

In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are needed in the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present specification, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.

Fig. 1 is a flow chart of a method for identifying an abnormal image according to an embodiment of the present application;

fig. 2 is a schematic structural diagram of an abnormal image recognition device according to an embodiment of the present application;

fig. 3 is a schematic structural diagram of an electronic device according to an embodiment of the present application.

[ detailed description ] of the invention

For a better understanding of the technical solutions of the present specification, the following detailed description of the embodiments of the present application is given with reference to the accompanying drawings.

It should be understood that the described embodiments are only some, but not all, of the embodiments of the present description. All other embodiments, which can be made by one of ordinary skill in the art based on the embodiments herein without making any inventive effort, are intended to be within the scope of the present disclosure.

The terminology used in the embodiments of the application is for the purpose of describing particular embodiments only and is not intended to be limiting of the description. As used in this application and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise.

With the development of aircraft technology, people have increasingly higher requirements on the performance of an aircraft, in the flight process of the aircraft, obstacles often need to be identified and classified, and the identification and classification of the obstacles are realized by largely relying on image transmission between an image sensor and an image processor in independent states, namely, the image sensor needs to send generated images to the image processor, and the image processor guides the subsequent flight process of the aircraft after identifying and classifying based on the received images.

The inventor of the application researches and discovers that during the flight of the aircraft, the image sensor can be abnormal in the process of acquiring images or in the process of transmitting images to the image processor, so that the Raw images sent to the image processor are abnormal, and the subsequent flight process of guiding the aircraft based on the abnormal Raw images can influence the flight safety of the aircraft. Therefore, how to identify an abnormal image from an image sensor is a problem to be solved.

In view of this, an embodiment of the present application provides a method for identifying an abnormal image, where in the method, for a plurality of Raw images generated by an image sensor, if the number of Raw images received at adjacent expected times is insufficient, it may be considered that an image loss or an inter-frame freeze occurs; if the pixel values of a plurality of adjacent pixel points in the same Raw image are the same, the intra-frame freezing abnormality can be considered to occur; if the number of the actual pixel points in the same Raw image is lower than the number of the target pixel points, the pixel points are considered to be lost abnormally; if the expected receiving time and the actual receiving time of the same Raw image have larger differences, the delay abnormality can be considered to occur; if the number of pixel points of the color channel which is absent in the same Raw image is large, channel missing abnormality can occur. If the at least one abnormal image is identified from the plurality of Raw images, alarm information can be output, and related staff can be informed of the abnormal type of the abnormal image, so that the related staff can take corresponding measures in time, and the flight safety of the aircraft is improved.

The following describes the embodiments of the present application in detail with reference to the drawings. Referring to fig. 1, an embodiment of the present application provides a method for identifying an abnormal image, which is applied to an image processor in an aircraft, and the flow of the method is described as follows:

step 101: a plurality of Raw images generated by an image sensor are received.

An image sensor and an image processor may be considered to be disposed within the aircraft, the image sensor may be considered to periodically acquire image data, sequentially generate Raw images, and transmit the generated Raw images to the image processor. After receiving the plurality of Raw images generated by the image sensor, the image processor can guide the flight process of the aircraft based on the received plurality of Raw images. For example, the received plurality of Raw images are directly identified and classified in sequence, or are identified and classified after being preprocessed to a certain degree, so that guidance is provided for the subsequent flight process of the aircraft.

Step 102: and determining the Raw image which meets the preset condition in the plurality of Raw images as an abnormal Raw image.

In this embodiment of the present application, for a plurality of Raw images generated by the image sensor, filtering may be performed based on a preset condition, so that a Raw image that satisfies the preset condition is determined as an abnormal Raw image.

For example, the preset condition may be: if the number of the Raw images received at the adjacent actual receiving moments is lower than a first set threshold, that is, the number of the Raw images received at the plurality of adjacent actual receiving moments does not meet the expected number, the image loss occurs in the Raw image transmission process or the inter-frame freezing occurs in the Raw image generation process.

If the pixel values of the adjacent pixels exceeding the second set threshold exist in the same Raw image, that is, the pixel values of the adjacent pixels existing in a larger number in the same Raw image are the same, it can be considered that intra-frame freezing occurs.

If the number of the actual pixels in the same Raw image is lower than the number of the target pixels, the pixel loss in the transmission process can be considered.

If the time difference between the expected receiving time and the actual receiving time of the same Raw image is greater than the third set threshold, that is, the actual receiving time is more than the expected receiving time, it can be considered that the image transmission process has overtime.

If the number of the pixels lacking the color channel in the same Raw image is greater than the fourth set threshold, that is, the number of the pixels lacking the color channel is greater, the generated Raw image can be considered to have color channel missing.

The following describes in detail how the above-described different types of abnormal images are determined.

Exception type one: inter-image frame freezing

In this embodiment of the present application, in a process that the image sensor sequentially generates the Raw images, it may be considered that each Raw image is given an image sequence number representing a generation sequence thereof, that is, the image data of the Raw image includes information of the image sequence number of the image sensor itself, and if the image sensor suddenly fails in a gap of generating adjacent Raw images, it may be caused that the image sensor does not generate a new Raw image at a subsequent time until the image sensor returns to normal. Therefore, if the actual change range of the image sequence numbers of the Raw images received at the adjacent actual reception times is smaller than the first set threshold, the Raw image received at the adjacent actual reception times is regarded as an abnormal Raw image, and the abnormal type of the abnormal Raw image is inter-frame freezing.

For example, the image sensor originally generates the Raw image n, the Raw image n+1, and the Raw image n+2 at times t-1, t, and t+1, respectively, when the first set threshold is set to 3, but if the image sensor itself fails after the Raw image n is generated and before the Raw image n+1 is generated, such that the image sensor does not generate the Raw image n+1 at time t when the shutter is pressed, and if the image sensor is restored to normal at time t+1, the Raw image n+1 is generated when the shutter is pressed again. That is, the actual change amplitude is (n+1) - (n-1) =2, which is smaller than the first set threshold, and the image processor knows that the Raw image received at three adjacent receiving moments has an inter-frame freezing problem in the generating process.

Anomaly type two: the pixel point is frozen in the frame.

In this embodiment of the present application, if a fault occurs in the image sensor during the process of generating any one of the plurality of Raw images, the pixels formed later are in a repeated state from the moment of occurrence of the fault, that is, the pixel values of the pixels formed later are the same. Therefore, after the image processor receives any one of the Raw images, if it is determined that the number of times that the same pixel value continuously appears in the pixel points at adjacent positions in the Raw image exceeds a second set threshold, it indicates that the Raw image is an abnormal Raw image, and the abnormal type of the abnormal Raw image is intra-frame freezing.

For example, the second set threshold is 10, and in the process of generating the 5 th Raw image, it can be considered that the generation is based on line pixels, i.e., the exposure process is performed line by line. For a Raw image including 100×100 pixels, if the image sensor suddenly fails during the exposure of the 50 th line, the pixels formed later from the 50 th line are all the same as the pixels generated before the image sensor fails, and correspondingly, the pixels formed later from the 50 th line are all the same as the pixels generated before the image sensor fails. For the image processor, after receiving the 5 th Raw image, if it is determined that the pixel values of more than 10 consecutive pixels appear from 50 lines are the same, the image processor knows that the 5 th Raw image has an intra-frame freezing problem in the generating process.

Abnormality type three: the image is lost between frames.

In the embodiment of the present application, in a process that the image sensor sequentially generates a plurality of Raw images, each Raw image may be considered to be given an image sequence number representing a generation sequence thereof, that is, the image data of the Raw image includes information of its own image sequence number. When the image sensor transmits the plurality of Raw images to the image processor respectively, a problem of partial Raw image loss may occur, that is, after the image processor receives the plurality of Raw images from the image sensor, if it is determined that a jump occurs between respective image serial numbers of the Raw images received at adjacent actual receiving moments, the Raw images received at the adjacent actual receiving moments may be considered as abnormal Raw images, and an abnormal type of the abnormal Raw images is inter-frame loss.

For example, if the image sensor originally generates the Raw image n, the Raw image n+1, and the Raw image n+2 at the time t-1, the time t, and the time t+1, respectively, the Raw image n+1 is lost during the process of transmitting the Raw image n, the Raw image n+1, and the Raw image n+2 to the image processor by the image sensor. Then for the image processor, at three adjacent receiving moments, the Raw image n, the Raw image n+1 and the Raw image n+2 should theoretically be received respectively, i.e. the change of the image sequence number is an incremental change in the form of +1, but the images actually received at the three adjacent receiving moments are: the sequence numbers of the Raw images n and the Raw images n+2 are changed from n to n+2, at this time, the image processor knows that the Raw images received at three adjacent receiving moments have the inter-frame loss problem in the transmission process, and the Raw images n and the Raw images n+2 are considered as abnormal images with the inter-frame loss.

Abnormality type four: the pixel point is lost in the frame.

In the embodiment of the present application, in the process of generating a plurality of Raw images by the image sensor, each pixel point set in each Raw image may be considered to be given a pixel point set sequence number representing the position of the pixel point set, that is, the image data of the Raw image includes information of the pixel point set sequence number of the pixel point set. When the image sensor transmits any one of the Raw images to the image processor, the problem that part of the pixel point sets are lost may occur, namely, after the image processor receives any one of the Raw images from the image sensor, if it is determined that the sequence numbers of the pixel point sets of each of the adjacent pixel point sets jump, the part of the pixel point sets are considered to be lost in the transmission process, so that the Raw image is taken as an abnormal Raw image, and the abnormal type of the abnormal Raw image is intra-frame loss.

For example, in the process of generating the 5 th Raw image by the image sensor, each pixel point set in the 5 th Raw image can be considered to be given a pixel point set serial number for representing the position of the pixel point set, and the 5 th Raw image contains 100×100 pixels, and each 100 pixels are regarded as a pixel point set, so that the pixel point set serial number is from 1 to 100. If the pixel point set with the serial number of 41-50 is lost in the image transmission process, for the image processor, the serial number of the pixel point set in the 5 th Raw image is continuous from 1-40, and continuous from 51-100, but the jump from the serial number 40 to the serial number 51 of the pixel point set exists, and at the moment, the image processor knows that the 5 th Raw image has the problem of intra-frame loss of the pixel point set in the transmission process.

Abnormality type five: the transmission times out.

In the embodiment of the present application, when the image sensor acquires image data to generate a Raw image, the image processor may be considered to need to send a configuration instruction to configure parameters of the image sensor, for example, the configuration instruction is used to configure a frame rate and an exposure time of the image sensor, where parameters of the image sensor that are configurable by the configuration instruction are not particularly limited. Firstly, the image processor may determine a time T1 when the image processor itself sends a configuration instruction to the image sensor, and then, based on T1 and a fixed delay period, may determine an expected receiving time of an i-th Raw image in the plurality of Raw images, where it is understood that the fixed delay period is positively related to i. Meanwhile, the actual receiving time of the ith Raw image can be obtained, and once the time difference between the actual receiving time and the expected receiving time of the ith Raw image is determined to be larger than a third set threshold value, the delay in the process of transmitting the ith Raw image can be considered to be higher, so that the ith Raw image is determined to be an abnormal Raw image, and the abnormal type of the abnormal Raw image is overtime.

It should be understood that the above-mentioned fixed delay period may be considered as a period of time required for the image sensor to operate a mechanism by itself from the time when the image sensor receives the configuration instruction, until the configuration instruction is validated, and then until the ith Raw image is generated.

Abnormality type six: color channels are missing.

In the embodiment of the present application, in the process of generating a plurality of Raw images by the image sensor, each pixel point in any Raw image may be considered to have a corresponding RGB value. For the image processor, after receiving any one of the Raw images, the actual RGB values corresponding to each pixel point in the Raw image can be obtained. And comparing the actual RGB value of each pixel point in the Raw image with the corresponding expected RGB value, and if the actual RGB value of the pixel point exceeding the fourth set threshold value is different from the expected RGB value, considering the Raw image as an abnormal Raw image, wherein the difference type of the abnormal Raw image is color channel deletion.

Step 103: and outputting alarm information, wherein the alarm information is used for prompting the abnormal type of the abnormal Raw image.

In the embodiment of the application, when the abnormal images in the received plurality of Raw images are determined, the abnormal types of the abnormal images can be informed to related staff, so that the related staff can take targeted countermeasures in time, and the flight safety of the aircraft is ensured.

As a possible implementation manner, the image processor may output alarm information for prompting an abnormality type of the abnormal Raw image that is currently present.

Exemplary:

output message type one (corresponding to exception type one): and informing staff of the abnormal image frozen between frames.

In the embodiment of the application, the image processor may output alarm information, where the alarm information is used to prompt that an abnormal Raw image frozen between frames exists.

Output message type two (corresponding to exception type two): and informing the staff of the abnormal image frozen in the frame.

In the embodiment of the application, the image processor may output alarm information, where the alarm information is used to prompt that there is an abnormal Raw image frozen in the frame.

Output message type three (corresponding to exception type three): and informing staff of the abnormal image with the inter-frame loss.

In the embodiment of the application, the image processor may output alarm information, where the alarm information is used to prompt that there is an abnormal Raw image lost between frames.

Output message type four (corresponding to exception type four): and informing staff of the abnormal image lost in the frame.

In the embodiment of the application, the image processor may output alarm information, where the alarm information is used to prompt that there is an abnormal Raw image lost in the frame.

Output message type five (corresponding to exception type five): and informing the staff of the abnormal image with the timeout.

In the embodiment of the application, the image processor can output alarm information, and the alarm information is used for prompting that an abnormal image of a timeout type exists.

Output message type six (corresponding to exception type six): and informing the staff of the abnormal image with the color channel missing.

In the embodiment of the application, the image processor may output alarm information, where the alarm information is used to prompt that there is an abnormal Raw image with a color channel missing type.

It should be understood that in the above embodiment, the output alarm information may be implemented by any one or a combination of a plurality of acousto-optic devices, and is not particularly limited herein.

Referring to fig. 2, based on the same inventive concept, an embodiment of the present application further provides an apparatus for identifying an abnormal image, where the apparatus includes: a receiving unit 201, a determining unit 202, and an output unit 203.

A receiving unit 201, configured to receive a plurality of Raw graphs generated by an image sensor;

a determining unit 202, configured to determine a Raw image that meets a preset condition in the plurality of Raw images as an abnormal Raw image, where the preset condition is that a number of Raw images received at adjacent expected receiving moments is lower than a first set threshold, and/or pixel values of adjacent pixels exceeding a second set threshold in the same Raw image are the same, and/or a number of actual pixels included in the same Raw image is lower than a number of target pixels, and/or a time difference between an expected receiving moment and an actual receiving moment of any Raw image is greater than a third set threshold, and/or a number of pixels lacking a color channel in the same Raw image is greater than a fourth set threshold;

and the output unit 203 is used for outputting alarm information, wherein the alarm information is used for prompting the abnormal type of the abnormal Raw image.

Optionally, the determining unit 202 is specifically configured to:

acquiring image sequence numbers of the Raw images received at adjacent actual receiving moments, wherein the image sequence numbers are used for representing the generation sequence of the Raw images;

if the actual change amplitude of the image sequence numbers in the adjacent actual receiving moments is smaller than the first set threshold, the Raw image received in the adjacent expected receiving moments is taken as an abnormal Raw image, wherein the abnormal type of the abnormal Raw image is inter-frame freezing.

Optionally, the determining unit 202 is specifically configured to:

and if the number of times that the same pixel value continuously appears in the pixel points at the adjacent positions in one row of the row images exceeds a second set threshold, taking the row images as abnormal row images, wherein the abnormal type of the abnormal row images is intra-frame freezing.

Optionally, the determining unit 202 is specifically configured to:

and if the image sequence numbers jump in the adjacent actual receiving moments, taking the Raw images received in the adjacent actual receiving moments as abnormal Raw images, wherein the abnormal type of the abnormal Raw images is inter-frame loss.

Optionally, the determining unit 202 is specifically configured to:

acquiring a pixel point set sequence number of a pixel point set in a Raw image, wherein the pixel point set sequence number is used for representing the position of the pixel point set in the Raw image;

and if the pixel point set sequence numbers of the adjacent pixel point sets in the Raw image jump, taking the Raw image as an abnormal Raw image, wherein the abnormal type of the abnormal Raw image is intra-frame loss.

Optionally, the determining unit 202 is specifically configured to:

determining expected receiving time of an ith Raw chart in a plurality of Raw images based on time T1 when a configuration instruction is sent to an image sensor end and fixed delay time, wherein the configuration instruction is used for configuring the frame rate and the exposure time of the image sensor, and the fixed delay time is positively related to i;

obtaining the actual receiving moment of the ith Raw image;

and if the time difference between the actual receiving time and the expected receiving time is larger than a third set threshold value, determining the ith Raw image as an abnormal Raw image, wherein the abnormal type of the abnormal Raw image is a timeout type.

Optionally, the determining unit 202 is specifically configured to:

acquiring an actual RGB value corresponding to each pixel point of a Raw image;

if the number of pixels in the Raw image, of which the actual RGB value is different from the expected RGB value, is greater than a fourth set threshold, determining the Raw image as an abnormal Raw image, wherein the abnormal type of the abnormal Raw image is color channel missing.

Optionally, the image data of the Raw image includes information of an image sequence number of the Raw image and/or information of a pixel point set sequence number.

Referring to fig. 3, based on the same inventive concept, an embodiment of the present application provides an electronic device, where the electronic device includes at least one processor 301, and the processor 301 is configured to execute a computer program stored in a memory, to implement the steps of the method for identifying an abnormal image as shown in fig. 1 provided in the embodiment of the present application.

Alternatively, the processor 301 may be a central processing unit, a specific ASIC, or one or more integrated circuits for controlling the execution of programs.

Optionally, the electronic device may further comprise a memory 302 coupled to the at least one processor 301, the memory 302 may comprise ROM, RAM and disk memory. The memory 302 is used for storing data required for the operation of the processor 301, i.e. instructions executable by the at least one processor 301, the at least one processor 301 performing the method as shown in fig. 1 by executing the instructions stored by the memory 302. Wherein the number of memories 302 is one or more. The memory 302 is shown in fig. 3, but it should be noted that the memory 302 is not an essential functional block, and is therefore shown in fig. 3 by a broken line.

The physical devices corresponding to the receiving unit 201, the determining unit 202, and the output unit 203 may be the aforementioned processor 301. The electronic device may be used to perform the method provided by the embodiment shown in fig. 1. Therefore, for the functions that can be implemented by each functional module in the electronic device, reference may be made to the corresponding description in the embodiment shown in fig. 1, which is not repeated.

Embodiments of the present application also provide a computer storage medium, where the computer storage medium stores computer instructions that, when executed on a computer, cause the computer to perform the method described in fig. 1.

The foregoing description of the preferred embodiments is provided for the purpose of illustration only and is not intended to limit the scope of the disclosure, since any modifications, equivalents, improvements, etc. that fall within the spirit and principles of the disclosure are intended to be included within the scope of the disclosure.

Claims

1. A method of identifying an abnormal image, the method comprising:

receiving a plurality of Raw images generated by an image sensor;

2. The method according to claim 1, wherein determining a Raw image meeting a preset condition among the plurality of Raw images as an abnormal Raw image includes:

3. The method according to claim 1, wherein determining a Raw image meeting a preset condition among the plurality of Raw images as an abnormal Raw image includes:

4. The method according to claim 1, wherein determining a Raw image meeting a preset condition among the plurality of Raw images as an abnormal Raw image includes:

5. The method according to claim 1, wherein determining a Raw image meeting a preset condition among the plurality of Raw images as an abnormal Raw image includes:

6. The method according to claim 1, wherein determining a Raw image meeting a preset condition among the plurality of Raw images as an abnormal Raw image includes:

obtaining the actual receiving time of the i-piece Raw image;

7. The method according to claim 1, wherein determining a Raw image meeting a preset condition among the plurality of Raw images as an abnormal Raw image includes:

8. The method according to any one of claims 2,4 and 5, wherein the image data of the Raw image includes information of an image sequence number of the Raw image and/or information of a pixel point set sequence number.

9. An abnormal image recognition apparatus, characterized in that the apparatus comprises:

a determining unit, configured to determine, as an abnormal Raw image, raw images that meet a preset condition in the plurality of Raw images, where the preset condition is that a number of Raw images received at adjacent expected receiving moments is lower than a first set threshold, and/or pixel values of adjacent pixels exceeding a second set threshold in the same Raw image are the same, and/or a number of actual pixels included in the same Raw image is lower than a number of target pixels, and/or a time difference between an expected receiving moment and an actual receiving moment of any Raw image is greater than a third set threshold, and/or a number of pixels lacking a color channel in the same Raw image is greater than a fourth set threshold;

10. An electronic device comprising at least one processor and a memory coupled to the at least one processor, the at least one processor being configured to implement the steps of the method of any of claims 1-8 when executing a computer program stored in the memory.