CN111343356A

CN111343356A - Image processing method, image processing apparatus, storage medium, and electronic device

Info

Publication number: CN111343356A
Application number: CN202010166917.4A
Authority: CN
Inventors: 陈标
Original assignee: Guangdong Oppo Mobile Telecommunications Corp Ltd
Current assignee: Guangdong Oppo Mobile Telecommunications Corp Ltd
Priority date: 2020-03-11
Filing date: 2020-03-11
Publication date: 2020-06-26
Also published as: WO2021179804A1

Abstract

The disclosure provides an image processing method, an image processing device, a computer readable storage medium and an electronic device, and relates to the technical field of image processing. The method comprises the following steps: when the camera collects an image, storing the collected image to be processed; when the image to be processed is saved, scanning the image to be processed synchronously to obtain the information of the image to be processed; or scanning the image to be processed within a preset time after the image to be processed is stored to obtain the information of the image to be processed; and marking the image to be processed according to the information of the image to be processed. The method and the device can process the image collected by the camera in time so as to rapidly acquire the effective information contained in the image.

Description

Image processing method, image processing apparatus, storage medium, and electronic device

Technical Field

The present disclosure relates to the field of image processing technologies, and in particular, to an image processing method, an image processing apparatus, a computer-readable storage medium, and an electronic device.

Background

With the rapid development of intelligent terminals, more and more functions are provided on terminal equipment. In order to meet the photographing requirement of a user, most terminal equipment is provided with a camera so as to acquire images. And the acquired image often contains much information. Therefore, it is necessary to process the image to acquire information.

In the prior art, processing an image is generally performed when a terminal device is in a state of screen saving charging, background running of an application program, or the like after image shooting is completed. However, in the above process, the problem of heat generation of the terminal device is often caused due to high resource occupancy and high power consumption of image processing, which affects the service life of the terminal device, and the image processing process is limited by the terminal state, so that it is difficult to obtain image information in time, and the flexibility is poor. In addition, if the background performs an image processing process when the terminal device runs other programs, the performance of the terminal device may be relatively high, and the user experience may be affected due to the problems of jamming or unsmooth operation.

Therefore, how to adopt a reasonable and effective image processing method is a problem to be solved urgently in the prior art.

It is to be noted that the information disclosed in the above background section is only for enhancement of understanding of the background of the present disclosure, and thus may include information that does not constitute prior art known to those of ordinary skill in the art.

Disclosure of Invention

The present disclosure provides an image processing method, an image processing apparatus, a computer-readable storage medium, and an electronic device, so as to at least improve the problem that the image processing process affects the performance of a terminal device in the prior art to a certain extent.

Additional features and advantages of the disclosure will be set forth in the detailed description which follows, or in part will be obvious from the description, or may be learned by practice of the disclosure.

According to a first aspect of the present disclosure, there is provided an image processing method including: acquiring an image to be processed acquired by a camera; when the image to be processed is collected or stored, the image to be processed is synchronously scanned to obtain the information of the image to be processed; and marking the image to be processed according to the information of the image to be processed.

According to a second aspect of the present disclosure, there is provided an image processing apparatus comprising: the image acquisition device is used for acquiring the image to be processed acquired by the camera; the image scanning device is used for synchronously scanning the image to be processed when the image to be processed is acquired or stored to obtain the information of the image to be processed; and the image marking module is used for marking the image to be processed according to the information of the image to be processed.

According to a third aspect of the present disclosure, there is provided a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements the above-described image processing method.

According to a fourth aspect of the present disclosure, there is provided an electronic device comprising: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to perform the above-described image processing method via execution of the executable instructions.

The technical scheme of the disclosure has the following beneficial effects:

according to the image processing method, the image processing device, the computer readable storage medium and the electronic equipment, acquiring an image to be processed, which is acquired by a camera; when an image to be processed is collected or stored, synchronously scanning the image to be processed to obtain information of the image to be processed; and marking the image to be processed according to the information of the image to be processed. On one hand, the acquired image to be processed can be scanned in time during acquisition or storage, so that the 'scanning while shooting' or 'scanning after shooting' in the image acquisition process is realized, the problem of high system working pressure caused by one-time scanning of accumulated scanning tasks by a system is avoided, and meanwhile, the system power consumption is reduced; on the other hand, the exemplary embodiment synchronously scans the acquired image to be processed, identifies and marks the information contained in the image, can quickly present the comprehensive information of the image for the user, improves the timeliness of image information generation, and improves user experience.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and together with the description, serve to explain the principles of the disclosure. It is to be understood that the drawings in the following description are merely exemplary of the disclosure, and that other drawings may be derived from those drawings by one of ordinary skill in the art without the exercise of inventive faculty.

Fig. 1 shows a schematic diagram of an electronic device of the present exemplary embodiment;

fig. 2 shows a flowchart of an image processing method of the present exemplary embodiment;

FIG. 3 illustrates a sub-flowchart of one image processing method of the present exemplary embodiment;

fig. 4 shows a block diagram of a configuration of an image processing apparatus of the present exemplary embodiment;

fig. 5 shows a schematic diagram of a computer-readable storage medium of the present exemplary embodiment.

Detailed Description

Example embodiments will now be described more fully with reference to the accompanying drawings. Example embodiments may, however, be embodied in many different forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the concept of example embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided to give a thorough understanding of embodiments of the disclosure. One skilled in the relevant art will recognize, however, that the subject matter of the present disclosure can be practiced without one or more of the specific details, or with other methods, components, devices, steps, and the like. In other instances, well-known technical solutions have not been shown or described in detail to avoid obscuring aspects of the present disclosure.

Furthermore, the drawings are merely schematic illustrations of the present disclosure and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and thus their repetitive description will be omitted. Some of the block diagrams shown in the figures are functional entities and do not necessarily correspond to physically or logically separate entities. These functional entities may be implemented in the form of software, or in one or more hardware modules or integrated circuits, or in different networks and/or processor devices and/or microcontroller devices.

Exemplary embodiments of the present disclosure provide an electronic device for implementing an image processing method. The electronic device comprises at least a processor and a memory for storing executable instructions of the processor, the processor being configured to perform the image processing method via execution of the executable instructions.

The electronic device may be implemented in various forms, and may include, for example, a mobile device such as a mobile phone, a tablet computer, a notebook computer, a Personal Digital Assistant (PDA), a wearable device, and a fixed device such as a desktop computer and a smart television. The following takes the terminal device 100 in fig. 1 as an example, and exemplifies the configuration of the electronic device. It will be appreciated by those skilled in the art that the configuration of figure 1 can also be applied to fixed type devices, in addition to components specifically intended for mobile purposes. In other embodiments, terminal device 100 may include more or fewer components than shown, or combine certain components, or split certain components, or a different arrangement of components. The illustrated components may be implemented in hardware, software, or a combination of software and hardware. The interface relationship between the components is only schematically shown, and does not constitute a structural limitation for the terminal device 100. In other embodiments, the terminal device 100 may also interface differently from fig. 1, or a combination of multiple interfaces.

As shown in fig. 1, the terminal device 100 may specifically include: the mobile terminal includes a processor 110, an internal memory 121, an external memory interface 122, a Universal Serial Bus (USB) interface 130, a charging management Module 140, a power management Module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication Module 150, a wireless communication Module 160, an audio Module 170, a speaker 171, a receiver 172, a microphone 173, an earphone interface 174, a sensor Module 180, a display 190, a camera Module 191, an indicator 192, a button 193, a Subscriber Identity Module (SIM) card interface 194, and the like. Wherein the sensor module 180 may include a depth sensor 1801, a pressure sensor 1802, and the like.

Processor 110 may include one or more processing units, such as: the Processor 110 may include an Application Processor (AP), a modem Processor, a Graphics Processor (GPU), an Image Signal Processor (ISP), a controller, a video codec, a Digital Signal Processor (DSP), a baseband Processor, and/or a Neural-Network Processing Unit (NPU), and the like. The different processing units may be separate devices or may be integrated into one or more processors.

The controller can generate an operation control signal according to the instruction operation code and the timing signal to complete the control of instruction fetching and instruction execution.

A memory may also be provided in processor 110 for storing instructions and data. The memory may store instructions for implementing six modular functions: detection instructions, connection instructions, information management instructions, analysis instructions, data transmission instructions, and notification instructions, and are controlled to be executed by the processor 110. In some implementations, the memory in the processor 110 is a cache memory. The memory may hold instructions or data that have just been used or recycled by the processor 110. If the processor 110 needs to reuse the instruction or data, it can be called directly from memory. Avoiding repeated accesses reduces the latency of the processor 110, thereby increasing the efficiency of the system.

In some implementations, the processor 110 may include one or more interfaces. The Interface may include an Integrated Circuit (I2C) Interface, an Inter-Integrated Circuit built-in audio (I2S) Interface, a Pulse Code Modulation (PCM) Interface, a Universal Asynchronous Receiver/Transmitter (UART) Interface, a Mobile Industry Processor Interface (MIPI), a General-purpose input/Output (GPIO) Interface, a Subscriber Identity Module (SIM) Interface, and/or a Universal Serial Bus (USB) Interface, etc. Connections are made with other components of the terminal device 100 through different interfaces.

The USB interface 130 is an interface conforming to the USB standard specification, and may specifically be a MiniUSB interface, a microsusb interface, a USB type c interface, or the like. The USB interface 130 may be used to connect a charger to charge the terminal device 100, may also be connected to an earphone to play audio through the earphone, and may also be used to connect the terminal device 100 to other electronic devices, such as a computer and a peripheral device.

The charging management module 140 is configured to receive charging input from a charger. The charger may be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 may receive charging input from a wired charger via the USB interface 130. In some wireless charging embodiments, the charging management module 140 may receive a wireless charging input through a wireless charging coil of the terminal device 100. The charging management module 140 may also supply power to the electronic device through the power management module 141 while charging the battery 142.

The power management module 141 is used for connecting the battery 142, the charging management module 140 and the processor 110. The power management module 141 receives the input of the battery 142 and/or the charging management module 140, and supplies power to the processor 110, the internal memory 121, the display screen 190, the camera module 191, the wireless communication module 160, and the like. The power management module 141 may also be used to monitor parameters such as battery capacity, battery cycle count, battery state of health (leakage, impedance), etc. In some other embodiments, the power management module 141 may also be disposed in the processor 110. In other embodiments, the power management module 141 and the charging management module 140 may be disposed in the same device.

The wireless communication function of the terminal device 100 may be implemented by the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, a modem processor, a baseband processor, and the like.

The antennas 1 and 2 are used for transmitting and receiving electromagnetic wave signals. Each antenna in terminal device 100 may be used to cover a single or multiple communication bands. Different antennas can also be multiplexed to improve the utilization of the antennas. For example: the antenna 1 may be multiplexed as a diversity antenna of a wireless local area network. In other embodiments, the antenna may be used in conjunction with a tuning switch.

The mobile communication module 150 may provide a solution including 2G/3G/4G/5G wireless communication applied on the terminal device 100. The mobile communication module 150 may include at least one filter, a switch, a power Amplifier, a Low Noise Amplifier (LNA), and the like. The mobile communication module 150 may receive the electromagnetic wave from the antenna 1, filter, amplify, etc. the received electromagnetic wave, and transmit the electromagnetic wave to the modem processor for demodulation. The mobile communication module 150 may also amplify the signal modulated by the modem processor, and convert the signal into electromagnetic wave through the antenna 1 to radiate the electromagnetic wave. In some embodiments, at least some of the functional modules of the mobile communication module 150 may be disposed in the processor 110. In some embodiments, at least some of the functional modules of the mobile communication module 150 may be disposed in the same device as at least some of the modules of the processor 110.

The modem processor may include a modulator and a demodulator. The modulator is used for modulating a low-frequency baseband signal to be transmitted into a medium-high frequency signal. The demodulator is used for demodulating the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then passes the demodulated low frequency baseband signal to a baseband processor for processing. The low frequency baseband signal is processed by the baseband processor and then transferred to the application processor. The application processor outputs a sound signal through an audio device (not limited to the speaker 171, the receiver 172, etc.) or displays an image or video through the display screen 190. In some embodiments, the modem processor may be a stand-alone device. In other embodiments, the modem processor may be separate from the processor 110 and may be disposed in the same device as the mobile communication module 150 or other functional modules.

The Wireless Communication module 160 may provide solutions for Wireless Communication applied to the terminal device 100, including Wireless Local Area Networks (WLANs) (e.g., Wireless Fidelity (Wi-Fi) Networks), Bluetooth (BT), Global Navigation Satellite System (GNSS), Frequency Modulation (FM), Near Field Communication (NFC), Infrared (IR), and the like. The wireless communication module 160 may be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, performs frequency modulation and filtering processing on electromagnetic wave signals, and transmits the processed signals to the processor 110. The wireless communication module 160 may also receive a signal to be transmitted from the processor 110, perform frequency modulation and amplification on the signal, and convert the signal into electromagnetic waves through the antenna 2 to radiate the electromagnetic waves.

The terminal device 100 implements a display function by the GPU, the display screen 190, the application processor, and the like. The GPU is a microprocessor that is coupled to a display screen 190 and an application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs that execute program instructions to generate or alter display information.

The display screen 190 is used to display images, video, and the like. The display screen 190 includes a display panel. The Display panel may be a Liquid Crystal Display (LCD), an Organic Light-Emitting Diode (OLED), an Active matrix Organic Light-Emitting Diode (Active-matrix Organic Light-Emitting Diode, AMOLED), a flexible Light-Emitting Diode (FLED), a miniature, a Micro-o led, a Quantum dot Light-Emitting Diode (QLED), or the like. In some embodiments, the terminal device 100 may include 1 or N display screens 190, where N is a positive integer greater than 1.

The terminal device 100 may implement a shooting function through the ISP, the camera module 191, the video codec, the GPU, the display screen 190, the application processor, and the like.

The ISP is used to process the data fed back by the camera module 191. For example, when a photo is taken, the shutter is opened, light is transmitted to the camera photosensitive element through the lens, the optical signal is converted into an electrical signal, and the camera photosensitive element transmits the electrical signal to the ISP for processing and converting into an image visible to naked eyes. The ISP can also carry out algorithm optimization on the noise, brightness and skin color of the image. The ISP can also optimize parameters such as exposure, color temperature and the like of a shooting scene. In some embodiments, the ISP may be provided in the camera module 191.

The camera module 191 is used to capture still images or videos. The object generates an optical image through the lens and projects the optical image to the photosensitive element. The photosensitive element may be a Charge Coupled Device (CCD) or a Complementary Metal-Oxide-Semiconductor (CMOS) phototransistor. The light sensing element converts the optical signal into an electrical signal, which is then passed to the ISP where it is converted into a digital image signal. And the ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into image signal in standard RGB, YUV and other formats. In some embodiments, the terminal device 100 may include 1 or N camera modules 191, where N is a positive integer greater than 1, and if the terminal device 100 includes N cameras, one of the N cameras is a main camera.

The digital signal processor is used for processing digital signals, and can process digital image signals and other digital signals. For example, when the terminal device 100 selects a frequency point, the digital signal processor is used to perform fourier transform or the like on the frequency point energy.

Video codecs are used to compress or decompress digital video. The terminal device 100 may support one or more video codecs. In this way, the terminal device 100 can play or record video in a plurality of encoding formats, such as: moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, MPEG4, and the like.

The external memory interface 122 may be used to connect an external memory card, such as a Micro SD card, to extend the storage capability of the terminal device 100. The external memory card communicates with the processor 110 through the external memory interface 122 to implement a data storage function. For example, files such as music, video, etc. are saved in an external memory card.

The internal memory 121 may be used to store computer-executable program code, which includes instructions. The internal memory 121 may include a program storage area and a data storage area. The storage program area may store an operating system, an application program (such as a sound playing function, an image playing function, etc.) required by at least one function, and the like. The storage data area may store data (such as audio data, a phonebook, etc.) created during use of the terminal device 100, and the like. In addition, the internal memory 121 may include a high-speed random access memory, and may further include a nonvolatile memory, such as at least one magnetic disk Storage device, a Flash memory device, a Universal Flash Storage (UFS), and the like. The processor 110 executes various functional applications of the terminal device 100 and data processing by executing instructions stored in the internal memory 121 and/or instructions stored in a memory provided in the processor.

The terminal device 100 may implement an audio function through the audio module 170, the speaker 171, the receiver 172, the microphone 173, the earphone interface 174, the application processor, and the like. Such as music playing, recording, etc.

The audio module 170 is used to convert digital audio information into an analog audio signal output and also to convert an analog audio input into a digital audio signal. The audio module 170 may also be used to encode and decode audio signals. The speaker 171, also called a "horn", converts an audio electric signal into a sound signal. A receiver 172, also called "earpiece", is used to convert the electrical audio signal into an acoustic signal. A microphone 173, also referred to as a "microphone", is used to convert sound signals into electrical signals. The earphone interface 174 is used to connect a wired earphone.

The depth sensor 1801 is used to acquire depth information of a scene. In some embodiments, the depth sensor may be disposed in the camera module 191.

The pressure sensor 1802 is used to sense a pressure signal, which can be converted into an electrical signal. In some embodiments, the pressure sensor 1802 may be disposed on the display screen 190. The pressure sensors 1802 can be of a wide variety, such as resistive pressure sensors, inductive pressure sensors, capacitive pressure sensors, and the like.

In addition, other functional sensors, such as a magnetic sensor, an acceleration sensor, a distance sensor, a proximity light sensor, a fingerprint sensor, a temperature sensor, a touch sensor, an ambient light sensor, a bone conduction sensor, etc., may be disposed in the sensor module 180 according to actual needs.

The keys 193 include a power-on key, a volume key, and the like. The keys 193 may be mechanical keys. Or may be touch keys. The terminal device 100 may receive a key input, and generate a key signal input related to user setting and function control of the terminal device 100.

Indicator 192 may be an indicator light that may be used to indicate a state of charge, a change in charge, or a message, missed call, notification, etc.

The SIM card interface 194 is used to connect a SIM card. The SIM card can be brought into and out of contact with the terminal device 100 by being inserted into the SIM card interface 194 or being pulled out from the SIM card interface 194. The terminal device 100 may support 1 or N SIM card interfaces, where N is a positive integer greater than 1. The SIM card interface 194 may support a Nano SIM card, a Micro SIM card, a SIM card, etc. Multiple cards can be inserted into the same SIM card interface 194 at the same time. The types of the plurality of cards may be the same or different. The SIM card interface 194 may also be compatible with different types of SIM cards. The SIM card interface 194 may also be compatible with an external memory card. The terminal device 100 interacts with the network through the SIM card to implement functions such as communication and data communication. In some embodiments, the terminal device 100 employs eSIM, namely: an embedded SIM card. The eSIM card may be embedded in the terminal device 100 and cannot be separated from the terminal device 100.

The exemplary embodiment of the present disclosure first provides an image processing method, which may be applied to a terminal device having a camera device, such as a mobile phone, a tablet computer, and a digital camera.

The following specifically describes an image processing method and an image processing apparatus according to exemplary embodiments of the present disclosure.

Fig. 2 shows a flow of an image processing method in the present exemplary embodiment, including the following steps S210 to S230:

and step S210, acquiring the image to be processed collected by the camera.

In a terminal device, one or more camera modules are usually configured, and the camera modules may include components such as a camera and an image sensor, and are used to collect and process raw image data to obtain an image to be processed. For example, in an intelligent terminal, a RAW image in RAW format may be acquired by a bayer image sensor, and subjected to "four-in-one" and demosaicing processing, or demosaicing and demosaicing processing, etc., to obtain a color image to be processed. In this exemplary embodiment, the image to be processed may be a preview image obtained by opening a camera by a user and by the camera, or may be a photographed image obtained after a photographing operation is performed by the user. The photographed image can be stored to the local, cloud or specific storage position in various forms according to user needs or storage requirements. For example, the image to be processed may be stored in an image format such as JPEG (Joint Photographic experts group), PNG (Portable Network Graphics), and the like, which is not specifically limited by the present disclosure.

Step S220, when the image to be processed is collected or stored, the image to be processed is synchronously scanned to obtain the information of the image to be processed.

And step S230, marking the image to be processed according to the information of the image to be processed.

In general, the image to be processed acquired by the camera includes many pieces of information, such as portrait information (number of people, sex of people, or others) when the object is a person, or environmental information (number and kind of trees or buildings, and location of landmark buildings) when the object is an environment. Therefore, it is necessary to scan the image to be processed to determine the information therein. The process of scanning the image to be processed is substantially a process of identifying information contained in the image to be processed, such as identifying whether or not there is a person, several persons, which person is, or a shooting environment in the image to be processed, and the like.

In an exemplary embodiment, scanning the image to be processed to obtain the information of the image to be processed may include the following steps:

extracting image characteristics of an image to be processed;

identifying the image characteristics to obtain the information of the image to be processed;

the information of the image to be processed includes any one or more of image classification information, person identification information, and object detection information.

The image feature refers to data capable of reflecting the feature of a subject in an image to be processed. The exemplary embodiment can identify the image to be processed by acquiring image features in the image to be processed and performing feature analysis on the image to be processed in a specific manner, such as a neural network model or other machine learning models, so as to determine one or more kinds of identification information.

The image to be processed can be divided into a plurality of types according to the difference of the shot objects, such as a landscape image, a person image, an animal image or a building image, and the like, and the information capable of reflecting the class attribute of the shot objects in the image to be processed is image classification information. The images of the terminal equipment can be effectively managed according to the image classification information, and a corresponding set is established for the images of the same class, so that convenience is provided for a user in image searching.

And the information of whether a person exists, several persons exist, which person exists, the sex of the person and the like in the image to be processed is the person identification information. The person identification information is determined, and the person images can be classified effectively, for example, images of the same person are classified into one group, or images related to a certain person are classified into one group and stored.

In addition, the image to be processed may include one or more objects to be photographed, such as mountains, water, food, buildings, etc., and may include only a person or a plurality of objects such as a person, a building, an animal, etc., in one image. Therefore, in the present exemplary embodiment, the information of the image to be processed may further include target detection information, which may reflect information of an object contained in the image. Such as which objects are included, how many objects are included, relationships or distances between different objects, etc.

Specifically, the present exemplary embodiment can implement the scanning of the image to be processed by any one of the following ways:

firstly, when an image to be processed is collected, the image to be processed is synchronously scanned;

the acquired to-be-processed image can be a preview image presented when the user opens the camera and does not perform the photographing operation, or a preview image temporarily cached after the user performs the photographing operation. The exemplary embodiment can perform timely information scanning on the preview image and cache the scanning result. After the image to be processed is determined, the scanning result of the preview image corresponding to the image to be processed can be directly used as the information of the image to be processed and is synchronously stored in the terminal together with the image to be processed, so that the efficiency of image scanning and information storage is effectively improved;

secondly, synchronously scanning the image to be processed when the image to be processed is saved;

generally, an image to be processed needs to be compressed when being saved, for example, when an image sensor processes a raw image, the obtained color image to be processed is compressed to be saved as an image in JPEG format. If the image to be processed is scanned after being stored for a long time, the image needs to be decompressed, which increases the workload of the processor or causes untimely information acquisition, thereby affecting the user experience. Therefore, the present exemplary embodiment can perform synchronous scanning on the image to be processed at the time of saving the image to be processed or at the time of completion of saving.

In the present exemplary embodiment, after the information of the image to be processed is obtained, the image to be processed may be marked according to the information. The marking may be a classification marking on the image to be processed, or a distinguishing marking on an object in the image to be processed. For example, the image to be processed may be divided into corresponding categories according to the image classification information; or classifying images of different person objects and images of task objects with incidence relations according to the person identification information; or marking the information such as age, sex, or possibly emotion of the person according to the target detection information. According to the embodiment, the mark is carried out according to the information of the image to be processed, so that a user can clearly browse and effectively search the image, and the use experience of the user is improved.

To sum up, in the present exemplary embodiment, an image to be processed acquired by a camera is acquired; when an image to be processed is collected or stored, synchronously scanning the image to be processed to obtain information of the image to be processed; and marking the image to be processed according to the information of the image to be processed. On one hand, the acquired image to be processed can be scanned in time during acquisition or storage, so that the 'scanning while shooting' or 'scanning after shooting' in the image acquisition process is realized, the problem of high system working pressure caused by one-time scanning of accumulated scanning tasks by a system is avoided, and meanwhile, the system power consumption is reduced; on the other hand, the exemplary embodiment synchronously scans the acquired image to be processed, identifies and marks the information contained in the image, can quickly present the comprehensive information of the image for the user, improves the timeliness of image information generation, and improves user experience.

In an exemplary embodiment, as shown in fig. 3, the step S210 may include the following steps:

step S310, acquiring a video stream collected by a camera;

step S320, selecting at least one frame of key frame image from the video stream as an image to be processed.

In practical applications, users often take dynamic video streams to record information in addition to still images. In order to record the information content contained in the video stream, the video stream also needs to be scanned. In the process of shooting and storing video streams, a camera usually collects images frame by frame and uploads the collected image frames to be stored as complete video streams. Therefore, in the present exemplary embodiment, the information of the video stream may be determined by scanning one or more frames of key frame images in the video stream captured by the camera. The method comprises the steps that key frame images are selected from videos and used for image processing to-be-processed images, at least one key frame image can be determined in various modes, for example, a preset frame number is determined, and the key frame images with the preset frame number are selected from video streams at random; or determining a preset time node of the key frame image, and using the image at the preset time node as the key frame image, such as when the video stream starts, when the video stream ends, and the like.

In an exemplary embodiment, the selecting at least one key frame image from the video stream may include:

and extracting key frame images from the video stream according to a preset interval frame number.

The preset interval frame number refers to a preset interval time between each key frame image, which can be determined according to user needs or algorithm diversity, for example, 5s interval time is set, and frame images arriving every 5s from the beginning in a video stream can be used as key frame images. The specific setting of the number of interval frames is preset, and this disclosure is not particularly limited thereto. According to the preset interval frame number, frame images with the same time interval in the video stream are obtained and are used as images to be processed for scanning, image information of each time node in the video can be captured well, and therefore the generated video stream information is more comprehensive.

When the user shoots a video stream, the user may run or move greatly, and the problem of image blurring and unsharpness is caused by camera shake. Therefore, in an exemplary embodiment, the above-mentioned selecting at least one key frame image from the video stream may include the following steps:

when video streams are collected, pose parameter variation between two adjacent frames is synchronously acquired;

for any frame in the video stream, if the pose parameter variation between the frame and the previous frame and the pose parameter variation between the frame and the next frame are both smaller than a preset threshold, determining the frame as a stable frame;

at least one key frame image is extracted from the stable frames of the video stream.

In the present exemplary embodiment, the pose parameters may be data capable of reflecting the position or movement of the camera, such as parameters of rotation, displacement, and the like, and the pose parameters may be measured by elements provided in the terminal device, such as by a gyro device. A stable frame refers to a frame image in a video stream that is relatively stable and clear. When the video stream is collected, for any frame in the video stream, when the pose parameter variation between the frame and the previous frame and the pose parameter variation between the frame and the next frame are both smaller than the preset threshold, it can be considered that the image of the current frame in the video stream is relatively stable and the image is relatively clear. Therefore, it may be determined as a stable frame, and then key frame images are extracted from the stable frame according to a preset rule, for example, a preset number of key frame images are set to be extracted, or key frame images are extracted at preset time intervals. The exemplary embodiment obtains a relatively stable frame image from the video stream, extracts at least one frame of key frame image from the stable frame, and performs image scanning, thereby ensuring the effectiveness of the scanned image and improving the accuracy of the image scanning.

Further, in an exemplary embodiment, after obtaining the information of the image to be processed, the image processing method may further include:

and generating the information of the video stream according to the information of the image to be processed.

After at least one frame of image to be processed in the video stream is scanned, the information of the video stream can be determined according to the scanning result of the image to be processed. The information of the video stream may refer to information of an object or content included in the video stream, such as an object appearing in the video stream, behavior information or identity information of the object, or landscape or architectural information appearing in the video stream. In the above-described embodiment, the image to be processed may include various types of information, such as image classification information, person identification information, object detection information, and the like. In the present exemplary embodiment, the information of the video stream may be generated from the information of each frame image obtained by scanning. Specifically, when each frame of image to be processed is scanned, the corresponding scanning result may be cached first, and when all the images to be processed are scanned, the image information is integrated, so that the information of the entire video stream can be obtained. For example, in a video stream, the frame image at the beginning is a character image, including a person at the a position, the image at the end is a character image of the same person, and the person position moves to the B position, the video stream can be considered as a motion video stream based on the person, such as walking or running, and the generated video stream information may include information that the type of the video stream is a character video, the video stream has several people, the sex of the people appears, and what actions may be performed. Based on the generated video stream information, the present exemplary embodiment may also mark the video stream to mark the video stream for highlight segments. The specific method can be to mark the whole video stream, such as a people video stream or a landscape video stream; or some segments in the video stream may be marked, for example, two people appear at time a, who respectively appears at time B, one person appears at time C, landmark buildings appear in the scene at time C, and the like. By marking the video stream, a user can acquire more comprehensive, accurate and effective information of the video stream when browsing or inquiring the video stream.

In an exemplary embodiment, step S220 may include the following steps:

when an image to be processed is collected, scanning at least one preview image corresponding to the image to be processed to obtain pre-identification information;

and when the image to be processed is stored, scanning the image to be processed according to the pre-identification information to obtain the information of the image to be processed.

Generally, after a user opens a camera, a preview image of a current shooting object or shooting scene is presented in a screen, so that the user can confirm whether the state of the shot object is proper or not. In the present exemplary embodiment, the preview image may be first scanned, and the pre-recognition information of the preview image may be determined. When the user saves the image to be processed, the image to be processed can be scanned according to the pre-identification information, and the information of the image to be processed is obtained. Compared with a photographed image, the photographed content of the preview image is often the same, but the preview image is smaller, the number of pixels is lower, the data size is smaller, and the preview image does not need to be locally stored for a long time. Therefore, the scanning based on the preview image in the exemplary embodiment can reduce the working pressure of the processor on the image processing and improve the image processing efficiency; on the other hand, the image to be processed is scanned according to the pre-identification information, and the information of the preview image is combined with the image to be processed, so that the accuracy of image scanning is improved.

Further, in an exemplary embodiment, the image processing method may further include:

when the camera collects the preview image, the preview image is scanned to obtain the pre-identification information of the preview image.

That is, when the terminal device opens the camera to preview the image, the preview image can be scanned in real time (i.e. in a mode of 'recognizing while previewing'), and the result of the scanned pre-recognition information is cached, and when the corresponding image to be processed needs to be stored, the information of the image to be processed can be determined according to the corresponding pre-recognition information.

Exemplary embodiments of the present disclosure also provide an image processing apparatus. As shown in fig. 4, the image processing apparatus 400 may include: an image acquisition module 410, configured to acquire an image to be processed acquired by a camera; the image scanning module 420 is configured to scan the image to be processed synchronously when the image to be processed is acquired or stored, so as to obtain information of the image to be processed; and an image marking module 430, configured to mark the image to be processed according to information of the image to be processed.

In an exemplary embodiment, the image acquisition module includes: the video stream acquisition unit is used for acquiring a video stream acquired by the camera; and the key frame selecting unit is used for selecting at least one frame of key frame image from the video stream as an image to be processed.

In an exemplary embodiment, the image processing apparatus further includes: and the video information generating module is used for generating the information of the video stream according to the information of the image to be processed after the information of the image to be processed is obtained.

In an exemplary embodiment, the key frame selecting module is configured to extract key frame images from the video stream according to a preset number of frame intervals.

In an exemplary embodiment, the key frame selecting module includes: the variable quantity acquiring unit is used for synchronously acquiring the pose parameter variable quantity between two adjacent frames when the video stream is acquired; a stable frame determining unit, configured to determine, for any frame in the video stream, if a pose parameter variation between the frame and a previous frame and a pose parameter variation between the frame and a next frame are both smaller than a preset threshold, the frame as a stable frame; and the key frame extraction unit is used for extracting at least one frame of key frame image from the stable frame of the video stream.

In an exemplary embodiment, an image scanning module includes: the device comprises a pre-recognition unit, a preview unit and a processing unit, wherein the pre-recognition unit is used for scanning at least one preview image corresponding to an image to be processed when the image to be processed is acquired to obtain pre-recognition information; and the image scanning unit is used for scanning the image to be processed according to the pre-identification information when the image to be processed is stored to obtain the information of the image to be processed.

In an exemplary embodiment, the image processing apparatus further includes: and the preview image scanning module is used for scanning the preview image to obtain the pre-identification information of the preview image when the camera collects the preview image.

In an exemplary embodiment, an image scanning module includes: the characteristic extraction unit is used for extracting the image characteristics of the image to be processed; the image recognition unit is used for recognizing the image characteristics to obtain the information of the image to be processed; the information of the image to be processed includes any one or more of image classification information, person identification information, and object detection information.

The specific details of each module in the above apparatus have been described in detail in the method section, and details that are not disclosed may refer to the method section, and thus are not described again.

As will be appreciated by one skilled in the art, aspects of the present disclosure may be embodied as a system, method or program product. Accordingly, various aspects of the present disclosure may be embodied in the form of: an entirely hardware embodiment, an entirely software embodiment (including firmware, microcode, etc.) or an embodiment combining hardware and software aspects that may all generally be referred to herein as a "circuit," module "or" system.

Exemplary embodiments of the present disclosure also provide a computer-readable storage medium having stored thereon a program product capable of implementing the above-described method of the present specification. In some possible embodiments, various aspects of the disclosure may also be implemented in the form of a program product including program code for causing a terminal device to perform the steps according to various exemplary embodiments of the disclosure described in the above-mentioned "exemplary methods" section of this specification, when the program product is run on the terminal device, for example, any one or more of the steps in fig. 2 or fig. 3 may be performed.

Referring to fig. 5, a program product 500 for implementing the above method according to an exemplary embodiment of the present disclosure is described, which may employ a portable compact disc read only memory (CD-ROM) and include program code, and may be run on a terminal device, such as a personal computer. However, the program product of the present disclosure is not limited thereto, and in this document, a readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.

The program product may employ any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples (a non-exhaustive list) of the readable storage medium include: an electrical connection having one or more wires, a portable disk, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

A computer readable signal medium may include a propagated data signal with readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A readable signal medium may also be any readable medium that is not a readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.

Program code embodied on a readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

Program code for carrying out operations for the present disclosure may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the user's computing device and partly on a remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user computing device through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computing device (e.g., through the internet using an internet service provider).

Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the disclosure disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of the disclosure following, in general, the principles of the disclosure and including such departures from the present disclosure as come within known or customary practice within the art to which the disclosure pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the disclosure being indicated by the following claims.

It will be understood that the present disclosure is not limited to the precise arrangements described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is to be limited only by the terms of the appended claims.

Claims

1. An image processing method, comprising:

acquiring an image to be processed acquired by a camera;

when the image to be processed is collected or stored, the image to be processed is synchronously scanned to obtain the information of the image to be processed;

and marking the image to be processed according to the information of the image to be processed.

2. The method of claim 1, wherein the acquiring the image to be processed captured by the camera comprises:

acquiring a video stream collected by a camera;

and selecting at least one frame of key frame image from the video stream as the image to be processed.

3. The method of claim 2, wherein after obtaining the information of the image to be processed, the method further comprises:

4. The method of claim 2, wherein said selecting at least one key frame image from said video stream comprises:

and extracting the key frame images from the video stream according to a preset interval frame number.

5. The method of claim 2, wherein said selecting at least one key frame image from said video stream comprises:

synchronously acquiring the pose parameter variation between two adjacent frames when the video stream is collected;

at least one key frame image is extracted from the stable frame of the video stream.

6. The method according to claim 1, wherein the synchronously scanning the image to be processed to obtain the information of the image to be processed when the image to be processed is acquired or when the image to be processed is saved comprises:

when the image to be processed is collected, scanning at least one preview image corresponding to the image to be processed to obtain pre-identification information;

and scanning the image to be processed according to the pre-identification information when the image to be processed is saved to obtain the information of the image to be processed.

7. The method of claim 6, further comprising:

and when the camera collects the preview image, scanning the preview image to obtain the pre-identification information of the preview image.

8. The method according to any one of claims 1 to 6, wherein the scanning the image to be processed to obtain the information of the image to be processed comprises:

extracting image features of the image to be processed;

the information of the image to be processed comprises any one or more of image classification information, person identification information and target detection information.

9. An image processing apparatus characterized by comprising:

the image acquisition module is used for acquiring an image to be processed acquired by the camera;

the image scanning module is used for synchronously scanning the image to be processed when the image to be processed is acquired or stored to obtain the information of the image to be processed;

and the image marking module is used for marking the image to be processed according to the information of the image to be processed.

10. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the method of any one of claims 1 to 8.

11. An electronic device, comprising:

a processor; and

a memory for storing executable instructions of the processor;

wherein the processor is configured to perform the method of any of claims 1 to 8 via execution of the executable instructions.