WO2021179804A1 - Procédé de traitement d'image, dispositif de traitement d'image, support de stockage et appareil électronique - Google Patents

Procédé de traitement d'image, dispositif de traitement d'image, support de stockage et appareil électronique Download PDF

Info

Publication number
WO2021179804A1
WO2021179804A1 PCT/CN2021/073289 CN2021073289W WO2021179804A1 WO 2021179804 A1 WO2021179804 A1 WO 2021179804A1 CN 2021073289 W CN2021073289 W CN 2021073289W WO 2021179804 A1 WO2021179804 A1 WO 2021179804A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
processed
information
video stream
frame
Prior art date
Application number
PCT/CN2021/073289
Other languages
English (en)
Chinese (zh)
Inventor
陈标
Original Assignee
Oppo广东移动通信有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Oppo广东移动通信有限公司 filed Critical Oppo广东移动通信有限公司
Publication of WO2021179804A1 publication Critical patent/WO2021179804A1/fr

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00—Details of television systems
    • H04N5/04—Synchronising
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N1/00—Scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission; Details thereof
    • H04N1/04—Scanning arrangements, i.e. arrangements for the displacement of active reading or reproducing elements relative to the original or reproducing medium, or vice versa
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/80—Camera processing pipelines; Components thereof
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30204—Marker

Definitions

  • the present disclosure relates to the field of image processing technology, and in particular to an image processing method, an image processing device, a computer-readable storage medium, and electronic equipment.
  • the image processing is usually performed after the image is taken, the terminal device is in a state of charging on the screen, or the application program is running in the background.
  • the terminal equipment will have heat problems, which will affect the service life of the terminal equipment, and the image processing process will be limited by the state of the terminal. , It is difficult to obtain the image information in time, and the flexibility is poor.
  • the terminal device performs image processing in the background while other programs are running, it will cause greater performance on the performance of the terminal device, causing problems such as stalling or unsmoothness, and affecting user experience.
  • the present disclosure provides an image processing method, an image processing device, a computer-readable storage medium, and an electronic device, thereby improving at least to a certain extent the problem that the image processing process in the prior art affects the performance of the terminal device.
  • an image processing method which includes: acquiring an image to be processed collected by a camera; The processed image is scanned to obtain the information of the image to be processed; and the image to be processed is marked according to the information of the image to be processed.
  • an image processing device including: an image acquisition device for acquiring an image to be processed collected by a camera; an image scanning device for acquiring the image to be processed or saving When the image to be processed is used, the image to be processed is scanned synchronously to obtain the information of the image to be processed; the image marking module is used to mark the image to be processed according to the information of the image to be processed.
  • a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, the above-mentioned image processing method is realized.
  • an electronic device including: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to execute the executable instructions Perform the image processing method described above.
  • the image to be processed is acquired by the camera; when the image to be processed is acquired, or when the image to be processed is saved, the image to be processed is scanned synchronously, Obtain the information of the image to be processed; and mark the image to be processed according to the information of the image to be processed.
  • this exemplary embodiment can scan the acquired images to be processed in time when they are being collected or saved, so as to realize the “scan while shooting” or “scan as soon as they are shot” during the image acquisition process, and avoid the system from accumulating
  • One-time scanning of the scanning task of the system causes the problem of high working pressure of the system and reduces the power consumption of the system;
  • this exemplary embodiment performs synchronous scanning on the acquired image to be processed, and recognizes the information contained in the image and Marking them can quickly present comprehensive image information to users, improve the timeliness of image information generation, and improve user experience.
  • FIG. 1 shows a schematic diagram of the electronic device of this exemplary embodiment
  • Fig. 2 shows a flowchart of an image processing method of this exemplary embodiment
  • Fig. 3 shows a sub-flow chart of an image processing method of this exemplary embodiment
  • Fig. 4 shows a structural block diagram of an image processing device of this exemplary embodiment
  • FIG. 5 shows a schematic diagram of the computer-readable storage medium of this exemplary embodiment.
  • Exemplary embodiments of the present disclosure provide an electronic device for implementing an image processing method.
  • the electronic device at least includes a processor and a memory.
  • the memory is used to store executable instructions of the processor, and the processor is configured to execute the image processing method by executing the executable instructions.
  • Electronic devices can be implemented in various forms. For example, they can include mobile devices such as mobile phones, tablet computers, notebook computers, personal digital assistants (PDAs), wearable devices, and stationary devices such as desktop computers and smart TVs.
  • PDAs personal digital assistants
  • the configuration in FIG. 1 can also be applied to fixed-type equipment.
  • the terminal device 100 may include more or fewer components than shown, or combine certain components, or split certain components, or arrange different components.
  • the illustrated components can be implemented in hardware, software, or a combination of software and hardware.
  • the interface connection relationship between the components is only shown schematically, and does not constitute a structural limitation of the terminal device 100.
  • the terminal device 100 may also adopt a different interface connection manner from that in FIG. 1 or a combination of multiple interface connection manners.
  • the terminal device 100 may specifically include: a processor 110, an internal memory 121, an external memory interface 122, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, Battery 142, antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, audio module 170, speaker 171, receiver 172, microphone 173, earphone interface 174, sensor module 180, display screen 190, camera module 191, indicator 192, buttons 193, Subscriber Identification Module (SIM) card interface 194, etc.
  • the sensor module 180 may include a depth sensor 1801, a pressure sensor 1802, and the like.
  • the processor 110 may include one or more processing units.
  • the processor 110 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), and an image signal processor. (Image Signal Processor, ISP), controller, video codec, digital signal processor (Digital Signal Processor, DSP), baseband processor and/or Neural-Network Processing Unit (NPU), etc.
  • AP application processor
  • modem processor GPU
  • image signal processor ISP
  • controller Video codec
  • DSP Digital Signal Processor
  • NPU Neural-Network Processing Unit
  • the different processing units may be independent devices or integrated in one or more processors.
  • the controller can generate operation control signals according to the instruction operation code and timing signals to complete the control of fetching instructions and executing instructions.
  • a memory may also be provided in the processor 110 to store instructions and data.
  • the memory can store instructions for implementing six modular functions: detection instructions, connection instructions, information management instructions, analysis instructions, data transmission instructions, and notification instructions, and the processor 110 controls the execution.
  • the memory in the processor 110 is a cache memory.
  • the memory can store instructions or data that have just been used or recycled by the processor 110. If the processor 110 needs to use the instruction or data again, it can be directly called from the memory. Repeated accesses are avoided, the waiting time of the processor 110 is reduced, and the efficiency of the system is improved.
  • the processor 110 may include one or more interfaces.
  • Interfaces can include integrated circuit (Inter-Integrated Circuit, I2C) interface, integrated circuit built-in audio (Inter-Integrated Circuit Sound, I2S) interface, pulse code modulation (Pulse Code Modulation, PCM) interface, universal asynchronous transceiver (Universal Asynchronous) Receiver/Transmitter, UART) interface, Mobile Industry Processor Interface (MIPI), General-Purpose Input/Output (GPIO) interface, Subscriber Identity Module (SIM) interface and/ Or Universal Serial Bus (Universal Serial Bus, USB) interface, etc. Connections are formed with other components of the terminal device 100 through different interfaces.
  • I2C integrated circuit
  • I2S integrated circuit built-in audio
  • PCM pulse code modulation
  • PCM Pulse Code Modulation
  • UART Universal Asynchronous transceiver
  • MIPI Mobile Industry Processor Interface
  • GPIO General-Purpose Input/Output
  • SIM Subscriber Identity Module
  • USB Universal Serial Bus
  • the USB interface 130 is an interface that complies with the USB standard specification, and specifically may be a MiniUSB interface, a MicroUSB interface, a USBTypeC interface, and the like.
  • the USB interface 130 can be used to connect a charger to charge the terminal device 100, or connect to earphones to play audio through the earphones, and can also be used to connect the terminal device 100 to other electronic devices, such as computers and peripherals.
  • the charging management module 140 is used to receive charging input from the charger.
  • the charger can be a wireless charger or a wired charger.
  • the charging management module 140 may receive the charging input of the wired charger through the USB interface 130.
  • the charging management module 140 may receive a wireless charging input through the wireless charging coil of the terminal device 100. While the charging management module 140 charges the battery 142, it can also supply power to the electronic device through the power management module 141.
  • the power management module 141 is used to connect the battery 142, the charging management module 140 and the processor 110.
  • the power management module 141 receives input from the battery 142 and/or the charging management module 140, and supplies power to the processor 110, the internal memory 121, the display screen 190, the camera module 191, and the wireless communication module 160.
  • the power management module 141 can also be used to monitor parameters such as battery capacity, battery cycle times, and battery health status (leakage, impedance). In some other implementation manners, the power management module 141 may also be provided in the processor 110. In other embodiments, the power management module 141 and the charging management module 140 may also be provided in the same device.
  • the wireless communication function of the terminal device 100 can be implemented by the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor, and the baseband processor.
  • the antenna 1 and the antenna 2 are used to transmit and receive electromagnetic wave signals.
  • Each antenna in the terminal device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve antenna utilization.
  • Antenna 1 can be multiplexed as a diversity antenna of a wireless local area network.
  • the antenna can be used in combination with a tuning switch.
  • the mobile communication module 150 may provide a wireless communication solution including 2G/3G/4G/5G and the like applied to the terminal device 100.
  • the mobile communication module 150 may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), and the like.
  • the mobile communication module 150 can receive electromagnetic waves by the antenna 1, filter, amplify, etc. the received electromagnetic waves, and transmit them to the modem processor for demodulation.
  • the mobile communication module 150 can also amplify the signal modulated by the modem processor, and convert it into electromagnetic waves for radiation via the antenna 1.
  • at least part of the functional modules of the mobile communication module 150 may be provided in the processor 110.
  • at least part of the functional modules of the mobile communication module 150 and at least part of the modules of the processor 110 may be provided in the same device.
  • the modem processor may include a modulator and a demodulator.
  • the modulator is used to modulate the low frequency baseband signal to be sent into a medium and high frequency signal.
  • the demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal.
  • the demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing.
  • the application processor outputs a sound signal through an audio device (not limited to the speaker 171, the receiver 172, etc.), or displays an image or video through the display screen 190.
  • the modem processor may be an independent device.
  • the modem processor may be independent of the processor 110 and be provided in the same device as the mobile communication module 150 or other functional modules.
  • the wireless communication module 160 can provide applications on the terminal device 100 including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), and global navigation satellites. System (Global Navigation Satellite System, GNSS), Frequency Modulation (FM), Near Field Communication (NFC), Infrared Technology (Infrared, IR) and other wireless communication solutions.
  • the wireless communication module 160 may be one or more devices integrating at least one communication processing module.
  • the wireless communication module 160 receives electromagnetic waves via the antenna 2, frequency modulates and filters the electromagnetic wave signals, and sends the processed signals to the processor 110.
  • the wireless communication module 160 may also receive a signal to be sent from the processor 110, perform frequency modulation, amplify, and convert it into electromagnetic waves to radiate through the antenna 2.
  • the terminal device 100 implements a display function through a GPU, a display screen 190, an application processor, and the like.
  • the GPU is a microprocessor, which is connected to the display screen 190 and the application processor.
  • the GPU is used to perform mathematical and geometric calculations and is used for graphics rendering.
  • the processor 110 may include one or more GPUs that execute program instructions to generate or change display information.
  • the display screen 190 is used to display images, videos, and the like.
  • the display screen 190 includes a display panel.
  • the display panel can be a Liquid Crystal Display (LCD), Organic Light-Emitting Diode (OLED), Active-Matrix Organic Light-Emitting Diode, or Active-Matrix Organic Light-Emitting Diode (Active-Matrix Organic Light-Emitting Diode).
  • Emitting Diode, AMOLED Flexible Light Emitting Diode (Flexlight-Emitting Diode, FLED), Miniled, MicroLed, Micro-oLed, Quantum Dot Light Emitting Diodes (QLED), etc.
  • the terminal device 100 may include one or N display screens 190, and N is a positive integer greater than one.
  • the terminal device 100 can implement a shooting function through an ISP, a camera module 191, a video codec, a GPU, a display screen 190, and an application processor.
  • the ISP is used to process the data fed back from the camera module 191. For example, when taking a picture, the shutter is opened, and the light is transmitted to the photosensitive element of the camera through the lens, the light signal is converted into an electrical signal, and the photosensitive element of the camera transmits the electrical signal to the ISP for processing, which is converted into an image visible to the naked eye.
  • ISP can also optimize the image noise, brightness, and skin color. ISP can also optimize the exposure, color temperature and other parameters of the shooting scene.
  • the ISP may be provided in the camera module 191.
  • the camera module 191 is used to capture still images or videos.
  • the object generates an optical image through the lens and is projected to the photosensitive element.
  • the photosensitive element may be a Charge Coupled Device (CCD) or a Complementary Metal-Oxide-Semiconductor (CMOS) phototransistor.
  • CCD Charge Coupled Device
  • CMOS Complementary Metal-Oxide-Semiconductor
  • the photosensitive element converts the optical signal into an electrical signal, and then transfers the electrical signal to the ISP to convert it into a digital image signal.
  • ISP outputs digital image signals to DSP for processing.
  • DSP converts digital image signals into standard RGB, YUV and other formats of image signals.
  • the terminal device 100 may include 1 or N camera modules 191, and N is a positive integer greater than 1. If the terminal device 100 includes N cameras, one of the N cameras is the main camera.
  • Digital signal processors are used to process digital signals. In addition to digital image signals, they can also process other digital signals. For example, when the terminal device 100 selects a frequency point, the digital signal processor is used to perform Fourier transform on the energy of the frequency point.
  • Video codecs are used to compress or decompress digital video.
  • the terminal device 100 may support one or more video codecs. In this way, the terminal device 100 can play or record videos in multiple encoding formats, such as: Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, MPEG4, and so on.
  • MPEG Moving Picture Experts Group
  • MPEG2 MPEG2, MPEG3, MPEG4, and so on.
  • the external memory interface 122 may be used to connect an external memory card, such as a Micro SD card, so as to expand the storage capacity of the terminal device 100.
  • the external memory card communicates with the processor 110 through the external memory interface 122 to realize the data storage function. For example, save music, video and other files in an external memory card.
  • the internal memory 121 may be used to store computer executable program code, and the executable program code includes instructions.
  • the internal memory 121 may include a storage program area and a storage data area.
  • the storage program area can store an operating system, an application program (such as a sound playback function, an image playback function, etc.) required by at least one function, and the like.
  • the data storage area can store data (such as audio data, phone book, etc.) created during the use of the terminal device 100.
  • the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, a universal flash storage (Universal Flash Storage, UFS), and the like.
  • the processor 110 executes various functional applications and data processing of the terminal device 100 by running instructions stored in the internal memory 121 and/or instructions stored in a memory provided in the processor.
  • the terminal device 100 can implement audio functions through an audio module 170, a speaker 171, a receiver 172, a microphone 173, a headphone interface 174, an application processor, and the like. For example, music playback, recording, etc.
  • the audio module 170 is used to convert digital audio information into an analog audio signal for output, and is also used to convert an analog audio input into a digital audio signal.
  • the audio module 170 can also be used to encode and decode audio signals.
  • the speaker 171, also called “speaker”, is used to convert audio electrical signals into sound signals.
  • the receiver 172, also called “earpiece”, is used to convert audio electrical signals into sound signals.
  • the microphone 173, also called “microphone” or “microphone”, is used to convert sound signals into electrical signals.
  • the earphone interface 174 is used to connect wired earphones.
  • the depth sensor 1801 is used to obtain depth information of the scene.
  • the depth sensor may be provided in the camera module 191.
  • the pressure sensor 1802 is used to sense pressure signals and can convert the pressure signals into electrical signals.
  • the pressure sensor 1802 may be provided on the display screen 190.
  • sensors with other functions can be set in the sensor module 180, such as magnetic sensors, acceleration sensors, distance sensors, proximity light sensors, fingerprint sensors, temperature sensors, touch sensors, ambient light sensors, bone conduction sensors, etc. Wait.
  • the button 193 includes a power-on button, a volume button, and so on.
  • the button 193 may be a mechanical button. It can also be a touch button.
  • the terminal device 100 may receive key input, and generate key signal input related to user settings and function control of the terminal device 100.
  • the indicator 192 may be an indicator light, which may be used to indicate the charging status, power change, or to indicate messages, missed calls, notifications, and so on.
  • the SIM card interface 194 is used to connect to a SIM card.
  • the SIM card can be inserted into the SIM card interface 194 or pulled out from the SIM card interface 194 to achieve contact and separation with the terminal device 100.
  • the terminal device 100 may support 1 or N SIM card interfaces, and N is a positive integer greater than 1.
  • the SIM card interface 194 can support Nano SIM cards, Micro SIM cards, SIM cards, etc.
  • the same SIM card interface 194 can insert multiple cards at the same time. The types of multiple cards can be the same or different.
  • the SIM card interface 194 can also be compatible with different types of SIM cards.
  • the SIM card interface 194 may also be compatible with external memory cards.
  • the terminal device 100 interacts with the network through the SIM card to implement functions such as call and data communication.
  • the terminal device 100 adopts an eSIM, that is, an embedded SIM card.
  • the eSIM card can be embedded in the terminal device 100 and cannot be separated from the terminal device 100.
  • Exemplary embodiments of the present disclosure first provide an image processing method, which can be applied to terminal devices with camera devices such as mobile phones, tablet computers, and digital cameras.
  • Fig. 2 shows the flow of an image processing method in this exemplary embodiment, including the following steps S210 to S220:
  • Step S210 Obtain the image to be processed collected by the camera.
  • the camera modules may include components such as cameras and image sensors for collecting raw image data and processing them to obtain images to be processed.
  • the original image in RAW format can be obtained through the Bayer image sensor, and it can be subjected to "four-in-one" and demosaic processing, or demosaic and demosaic processing, etc., to obtain a color to-be-processed image.
  • the image to be processed may be a preview image obtained by the user opening the camera and the camera, or may be a photographed image obtained after the user performs a photographing operation.
  • the photographed images can be saved to local, cloud or specific storage locations in various forms according to user needs or storage requirements.
  • the image to be processed may be stored in image formats such as JPEG (Joint Photographic Experts Group), PNG (Portable Network Graphics, portable network graphics), which is not specifically limited in the present disclosure.
  • JPEG Joint Photographic Experts Group
  • PNG Portable Network Graphics, portable network graphics
  • step S220 when the image to be processed is collected, or when the image to be processed is saved, the image to be processed is scanned synchronously to obtain the information of the image to be processed.
  • Step S230 Mark the image to be processed according to the information of the image to be processed.
  • the image to be processed by the camera will contain a lot of information, such as portrait information when the subject is a person (the number of people, gender, etc.), or environmental information when the subject is the environment (trees or buildings). Quantity, type, location of landmarks, etc.) etc. Therefore, it is necessary to scan the image to be processed to determine the information in it.
  • the process of scanning the image to be processed is essentially the process of recognizing the information contained in the image to be processed, such as identifying whether there are people in the image to be processed, how many people are there, which person it is, or the shooting in the image to be processed Environment and so on.
  • scanning the image to be processed to obtain information of the image to be processed may include the following steps:
  • the information of the image to be processed includes any one or more of image classification information, person identification information, and target detection information.
  • Image feature refers to the data that can reflect the feature of the subject in the image to be processed.
  • the image features in the image to be processed can be obtained by using a specific method, such as a neural network model or other machine learning models, to perform feature analysis on the image to be processed, and to identify one of them. Or a variety of identifying information.
  • the images to be processed can be divided into multiple types, such as landscape images, human images, animal images, or architectural images, etc.
  • the information that can reflect the category attributes of the subject in the image to be processed is the image Classified information.
  • the image classification information the images of the terminal equipment can be effectively managed, and a corresponding set of images of the same type can be established, which provides convenience for users when searching for images.
  • Determining person identification information can effectively classify person images, such as grouping images of the same person into one category, or grouping images related to a person into one category for storage.
  • the image to be processed may include one or more subjects, such as mountains, water, food, or buildings.
  • An image may include only people or multiple objects such as people, buildings, and animals. . Therefore, in this exemplary embodiment, the information of the image to be processed may also include target detection information, and the target detection information may reflect the information of the object contained in the image. For example, which objects are included, how many objects are included, and the relationship or distance between different objects.
  • this exemplary embodiment may implement scanning of the image to be processed in any of the following ways:
  • the first is to scan the image to be processed synchronously when acquiring the image to be processed
  • the collected image to be processed may refer to the preview image presented when the user turns on the camera without taking a photograph, or it may be a temporarily cached preview image after the user performs the photographing operation.
  • This exemplary embodiment can perform timely information scanning on the preview image, and cache the scanning result.
  • the scanning result of the preview image corresponding to the image to be processed can be directly used as the information of the image to be processed, and stored in the terminal synchronously with the image to be processed, effectively improving the efficiency of image scanning and information storage;
  • the second type is to scan the image to be processed synchronously when saving the image to be processed
  • the image to be processed needs to be compressed when it is saved.
  • the image sensor processes the original image, it compresses the obtained color image to be processed to save it as an image in JPEG format. If the image to be processed is scanned after being saved for a long time, it needs to be decompressed. This process will increase the workload of the processor, or cause the information to be obtained in time, which affects the user experience. Therefore, this exemplary embodiment can perform synchronous scanning of the image to be processed when the image to be processed is saved or when the saving is completed.
  • the image to be processed can be marked according to the information.
  • the labeling may refer to the processing of classifying and marking the image to be processed, or the processing of distinguishing and marking objects in the image to be processed.
  • the images to be processed can be classified into corresponding categories according to the image classification information; or according to the person recognition information, images of different person objects and images of task objects with an association relationship can be classified; or they can be processed according to the target detection information
  • the portraits appearing in the images are marked with information such as age, gender, or the emotions that the characters may appear in.
  • the user can browse and effectively search the image in a clear manner, thereby improving the user experience.
  • the image to be processed collected by the camera is acquired; when the image to be processed is acquired, or when the image to be processed is saved, the image to be processed is scanned synchronously to obtain the information of the image to be processed; Information about the image to be processed, and mark the image to be processed.
  • this exemplary embodiment can scan the acquired images to be processed in time when they are being collected or saved, so as to realize the “scan while shooting” or “scan as soon as they are shot” during the image acquisition process, and avoid the system from accumulating
  • One-time scanning of the scanning task of the system causes the problem of high working pressure of the system and reduces the power consumption of the system;
  • this exemplary embodiment performs synchronous scanning on the acquired image to be processed, and recognizes the information contained in the image and Marking them can quickly present comprehensive image information to users, improve the timeliness of image information generation, and improve user experience.
  • step S210 may include the following steps:
  • Step S310 Obtain the video stream collected by the camera
  • Step S320 Select at least one key frame image from the video stream as the image to be processed.
  • the information of the video stream can be determined by scanning one or more key frame images in the video stream collected by the camera.
  • the key frame image is selected from the video and used for image processing to be processed.
  • At least one key frame image can be determined in a variety of ways, such as determining the preset number of frames, and randomly selecting the pre-processed image from the video stream. Set the key frame image of the number of frames; or determine the preset time node of the key frame image, and use the image at the preset time node as the key frame image, such as the beginning of the video stream and the end of the video stream.
  • the foregoing selecting at least one key frame image from the video stream may include:
  • the preset interval frame number refers to the preset interval time between each key frame image, which can be determined according to user needs or algorithm diversity. For example, if an interval time of 5s is set, then in a video stream from the beginning, every The frame image arriving in 5s can be used as the key frame image.
  • the specific setting of the preset interval frame number is not specifically limited in the present disclosure. According to the preset interval frame number, the frame images with the same time interval in the video stream are obtained and scanned as the image to be processed, which can better capture the image information of each time node in the video, so that the generated video stream information is more For comprehensive.
  • the foregoing selecting at least one key frame image from the video stream may include the following steps:
  • any frame in the video stream if the change in pose parameter between it and the previous frame and the change in pose parameter between it and the next frame are less than the preset threshold, then any frame is determined as Stable frame
  • the pose parameters may be data that can reflect the position or movement of the camera, such as parameters such as rotation, displacement, etc.
  • the pose parameters may be measured by components configured in the terminal device, for example, through a gyroscope device. The above parameters are measured.
  • a stable frame refers to a relatively stable and clear frame image in a video stream. When capturing a video stream, for any frame in the video stream, when the change in pose parameter between it and the previous frame and the change in pose parameter between it and the next frame are all less than the preset threshold, you can It is believed that the current frame image in the video stream is relatively stable and the image is relatively clear.
  • the key frame images can be extracted from the stable frame according to preset rules, such as setting to extract a preset number of key frame images, or extracting key frame images at a preset time interval, etc. .
  • This exemplary embodiment first obtains a relatively stable frame image from the video stream, and then extracts at least one key frame image from the stable frame, and performs image scanning to ensure the validity of the scanned image and improve the accuracy of image scanning. .
  • the image processing method may further include:
  • the information of the video stream can be determined according to the scanning result of the image to be processed.
  • the information of the video stream may refer to the information of the object or content included in the video stream, such as objects appearing in the video stream, behavior information or identity information of the objects, etc., or scenery or building information appearing in the video stream.
  • the image to be processed may include multiple types of information, such as image classification information, person recognition information, target detection information, and so on.
  • the information of the video stream can be generated based on the information of each frame of image obtained by scanning.
  • the corresponding scanning result can be cached first, and when all the to-be-processed images are scanned, the image information is integrated to obtain the information of the entire video stream.
  • the frame image at the beginning is a character image, including a person at position A
  • the end image is a character image of the same person. If the person’s position moves to position B, the video stream can be considered to be based on The person’s mobile video stream, such as an action video stream such as walking or running.
  • the generated video stream information can include, the type of the video stream is a character video, how many people appear in the video stream, the gender of the character, and what may be happening Actions and other information.
  • this exemplary embodiment may also mark the video stream to mark the video stream as a highlight. Specifically, it can mark the entire video stream, such as a character video stream or a landscape video stream; it can also mark certain segments in the video stream, for example, two people appear at time A, who are they, and who appear at time B For one person, landmark buildings and so on appeared on the screen at time C. By marking the video stream, users can obtain more comprehensive, accurate and effective video stream information when browsing or querying the video stream.
  • step S220 may include the following steps:
  • At least one preview image corresponding to the image to be processed is scanned to obtain pre-identification information
  • the image to be processed is scanned according to the pre-identification information to obtain the information of the image to be processed.
  • a preview image of the current subject or shooting scene will be displayed on the screen for the user to confirm whether the state of the subject is appropriate.
  • the preview image may be scanned first to determine the pre-identification information of the preview image.
  • the image to be processed can be scanned according to the pre-identification information to obtain the information of the image to be processed.
  • the content of the preview image is often the same, but the preview image is smaller, the number of pixels is lower, and the amount of data is smaller, and the preview image does not need to be stored locally for a long time. Therefore, this exemplary embodiment scans based on the preview image, which can reduce the working pressure of the processor on image processing and improve the efficiency of image processing; The combination of processing images improves the accuracy of image scanning.
  • the image processing method may further include:
  • the camera When the camera collects the preview image, it scans the preview image to obtain the pre-identification information of the preview image.
  • the preview image can be scanned in real time (that is, the "recognition while previewing" method), and the result of the scanned pre-identification information is cached.
  • the information of the image to be processed can be determined according to the corresponding pre-identification information.
  • the image processing device 400 may include: an image acquisition module 410 for acquiring images to be processed collected by a camera; and an image scanning module 420 for acquiring images to be processed or saving images to be processed When the time, the image to be processed is scanned synchronously to obtain the information of the image to be processed; the image marking module 430 is configured to mark the image to be processed according to the information of the image to be processed.
  • the image acquisition module includes: a video stream acquisition unit for acquiring a video stream collected by a camera; a key frame selection unit for selecting at least one key frame image from the video stream as the to-be-processed image.
  • the image processing device further includes: a video information generating module, configured to generate video stream information according to the information of the image to be processed after obtaining the information of the image to be processed.
  • a video information generating module configured to generate video stream information according to the information of the image to be processed after obtaining the information of the image to be processed.
  • the key frame selection module is configured to extract key frame images from the video stream according to the number of frames at preset intervals.
  • the key frame selection module includes: a change amount acquisition unit for synchronously acquiring the change amount of the pose parameter between two adjacent frames when the video stream is captured; a stable frame determination unit for For any frame in the video stream, if the change in pose parameter between it and the previous frame and the change in pose parameter between it and the next frame are less than the preset threshold, then any frame is determined as Stable frame:
  • the key frame extraction unit is used to extract at least one key frame image from the stable frame of the video stream.
  • the image scanning module includes: a pre-identification unit, configured to scan at least one preview image corresponding to the image to be processed to obtain pre-identification information when the image to be processed is collected; When saving the image to be processed, the image to be processed is scanned according to the pre-identification information to obtain the information of the image to be processed.
  • the image processing device further includes: a preview image scanning module, configured to scan the preview image when the camera collects the preview image to obtain pre-identification information of the preview image.
  • a preview image scanning module configured to scan the preview image when the camera collects the preview image to obtain pre-identification information of the preview image.
  • the image scanning module includes: a feature extraction unit for extracting image features of the image to be processed; an image recognition unit for recognizing image features to obtain information about the image to be processed; and image to be processed
  • the information includes any one or more of image classification information, person recognition information, and target detection information.
  • Exemplary embodiments of the present disclosure also provide a computer-readable storage medium on which is stored a program product capable of implementing the above-mentioned method of this specification.
  • various aspects of the present disclosure can also be implemented in the form of a program product, which includes program code.
  • the program product runs on a terminal device, the program code is used to make the terminal device execute the above-mentioned instructions in this specification.
  • the steps described in the "Exemplary Methods" section according to various exemplary embodiments of the present disclosure for example, any one or more steps in FIG. 2 or FIG. 3 may be performed.
  • a program product 500 for implementing the above method according to an exemplary embodiment of the present disclosure is described.
  • the program product of the present disclosure is not limited thereto.
  • the readable storage medium can be any tangible medium that contains or stores a program, and the program can be used by or in combination with an instruction execution system, device, or device.
  • the program product can adopt any combination of one or more readable media.
  • the readable medium may be a readable signal medium or a readable storage medium.
  • the readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or a combination of any of the above. More specific examples (non-exhaustive list) of readable storage media include: electrical connections with one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable Type programmable read only memory (EPROM or flash memory), optical fiber, portable compact disk read only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination of the above.
  • the computer-readable signal medium may include a data signal propagated in baseband or as a part of a carrier wave, and readable program code is carried therein. This propagated data signal can take many forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the foregoing.
  • the readable signal medium may also be any readable medium other than a readable storage medium, and the readable medium may send, propagate, or transmit a program for use by or in combination with the instruction execution system, apparatus, or device.
  • the program code contained on the readable medium can be transmitted by any suitable medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the above.
  • the program code for performing the operations of the present disclosure can be written in any combination of one or more programming languages.
  • the programming languages include object-oriented programming languages—such as Java, C++, etc., as well as conventional procedural programming. Language-such as "C" language or similar programming language.
  • the program code can be executed entirely on the user's computing device, partly on the user's device, executed as an independent software package, partly on the user's computing device and partly executed on the remote computing device, or entirely on the remote computing device or server Executed on.
  • the remote computing device can be connected to a user computing device through any kind of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (for example, using Internet service providers). Business to connect via the Internet).
  • LAN local area network
  • WAN wide area network
  • Internet service providers for example, using Internet service providers

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Studio Devices (AREA)

Abstract

La présente invention se rapporte au domaine technique du traitement d'image. La présente invention concerne un procédé de traitement d'image, un dispositif de traitement d'image, un support de stockage lisible par ordinateur et un appareil électronique. Le procédé consiste : lorsqu'une image à traiter est capturée par une caméra, à stocker l'image capturée ; à balayer de manière synchrone l'image tout en la stockant, et à obtenir des informations de l'image ; ou à balayer l'image dans un intervalle de temps prédéfini après que l'image a été stockée, et à obtenir des informations de l'image ; et à marquer l'image en fonction des informations de l'image. La présente divulgation permet à une image capturée par une caméra d'être traitée en temps opportun, de telle sorte que des informations effectives contenues dans ladite image peuvent être obtenues rapidement.
PCT/CN2021/073289 2020-03-11 2021-01-22 Procédé de traitement d'image, dispositif de traitement d'image, support de stockage et appareil électronique WO2021179804A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202010166917.4A CN111343356A (zh) 2020-03-11 2020-03-11 图像处理方法、图像处理装置、存储介质与电子设备
CN202010166917.4 2020-03-11

Publications (1)

Publication Number Publication Date
WO2021179804A1 true WO2021179804A1 (fr) 2021-09-16

Family

ID=71186272

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/073289 WO2021179804A1 (fr) 2020-03-11 2021-01-22 Procédé de traitement d'image, dispositif de traitement d'image, support de stockage et appareil électronique

Country Status (2)

Country Link
CN (1) CN111343356A (fr)
WO (1) WO2021179804A1 (fr)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114170553A (zh) * 2021-12-09 2022-03-11 北京字节跳动网络技术有限公司 图像处理方法、装置和电子设备
CN115065868A (zh) * 2022-04-12 2022-09-16 北京字节跳动网络技术有限公司 图像处理方法、装置、设备、系统及介质

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111343356A (zh) * 2020-03-11 2020-06-26 Oppo广东移动通信有限公司 图像处理方法、图像处理装置、存储介质与电子设备
CN114038197B (zh) * 2021-11-24 2023-06-13 浙江大华技术股份有限公司 场景状态的确定方法、装置、存储介质及电子装置

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101753823A (zh) * 2008-12-18 2010-06-23 三星电子株式会社 自动标记图像的装置及其方法
US20110320964A1 (en) * 2010-06-28 2011-12-29 International Business Machines Corporation Real-time preview of uniform resource identifier addressable dynamic content
CN104537662A (zh) * 2014-12-24 2015-04-22 百度在线网络技术（北京）有限公司 一种用于提供拍照图像的方法与设备
CN105518679A (zh) * 2015-03-26 2016-04-20 北京旷视科技有限公司 图片管理方法以及图片同步方法
CN107180074A (zh) * 2017-03-31 2017-09-19 北京奇艺世纪科技有限公司 一种视频分类方法及装置
CN108090126A (zh) * 2017-11-14 2018-05-29 维沃移动通信有限公司 图像处理方法、装置及移动终端、图像识别方法及服务器
CN109348120A (zh) * 2018-09-30 2019-02-15 烽火通信科技股份有限公司 一种拍摄方法、图像的显示方法、系统及设备
CN111343356A (zh) * 2020-03-11 2020-06-26 Oppo广东移动通信有限公司 图像处理方法、图像处理装置、存储介质与电子设备

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109670386A (zh) * 2017-10-16 2019-04-23 深圳泰首智能技术有限公司 人脸识别方法及终端
CN107734260A (zh) * 2017-10-26 2018-02-23 维沃移动通信有限公司 一种图像处理方法及移动终端
CN108259758B (zh) * 2018-03-18 2020-10-09 Oppo广东移动通信有限公司 图像处理方法、装置、存储介质和电子设备
CN109348125B (zh) * 2018-10-31 2020-02-04 Oppo广东移动通信有限公司 视频校正方法、装置、电子设备和计算机可读存储介质
CN110188718B (zh) * 2019-06-04 2021-04-02 南京大学 一种基于关键帧和联合稀疏表示的无约束人脸识别方法

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101753823A (zh) * 2008-12-18 2010-06-23 三星电子株式会社 自动标记图像的装置及其方法
US20110320964A1 (en) * 2010-06-28 2011-12-29 International Business Machines Corporation Real-time preview of uniform resource identifier addressable dynamic content
CN104537662A (zh) * 2014-12-24 2015-04-22 百度在线网络技术（北京）有限公司 一种用于提供拍照图像的方法与设备
CN105518679A (zh) * 2015-03-26 2016-04-20 北京旷视科技有限公司 图片管理方法以及图片同步方法
CN107180074A (zh) * 2017-03-31 2017-09-19 北京奇艺世纪科技有限公司 一种视频分类方法及装置
CN108090126A (zh) * 2017-11-14 2018-05-29 维沃移动通信有限公司 图像处理方法、装置及移动终端、图像识别方法及服务器
CN109348120A (zh) * 2018-09-30 2019-02-15 烽火通信科技股份有限公司 一种拍摄方法、图像的显示方法、系统及设备
CN111343356A (zh) * 2020-03-11 2020-06-26 Oppo广东移动通信有限公司 图像处理方法、图像处理装置、存储介质与电子设备

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114170553A (zh) * 2021-12-09 2022-03-11 北京字节跳动网络技术有限公司 图像处理方法、装置和电子设备
CN115065868A (zh) * 2022-04-12 2022-09-16 北京字节跳动网络技术有限公司 图像处理方法、装置、设备、系统及介质

Also Published As

Publication number Publication date
CN111343356A (zh) 2020-06-26

Similar Documents

Publication Publication Date Title
WO2021179804A1 (fr) Procédé de traitement d'image, dispositif de traitement d'image, support de stockage et appareil électronique
WO2021213120A1 (fr) Procédé et appareil de projection d'écran et dispositif électronique
WO2020078299A1 (fr) Procédé permettant de traiter un fichier vidéo et dispositif électronique
US20240205535A1 (en) Photographing method and electronic device
CN112449120B (zh) 高动态范围视频生成方法及装置
WO2021104485A1 (fr) Procédé de photographie et dispositif électronique
US10181203B2 (en) Method for processing image data and apparatus for the same
WO2021143269A1 (fr) Procédé photographique dans un scénario à longue distance focale, et terminal mobile
WO2021036318A1 (fr) Procédé de traitement d'image vidéo, et dispositif
CN111179282A (zh) 图像处理方法、图像处理装置、存储介质与电子设备
CN112492193B (zh) 一种回调流的处理方法及设备
WO2020173379A1 (fr) Procédé et dispositif de groupement de photographies
CN115835010A (zh) 一种拍摄方法及终端
CN112954251B (zh) 视频处理方法、视频处理装置、存储介质与电子设备
CN111694978A (zh) 图像相似度检测方法、装置、存储介质与电子设备
WO2021115483A1 (fr) Procédé de traitement d'image et appareil associé
WO2022022319A1 (fr) Procédé et système de traitement d'image, dispositif électronique et système de puce
CN115526787A (zh) 视频处理方法和装置
CN113436576A (zh) 应用于二维码扫描的oled显示屏调光方法及装置
CN115115679A (zh) 一种图像配准方法及相关设备
CN114429495A (zh) 一种三维场景的重建方法和电子设备
WO2020216096A1 (fr) Procédé de montage vidéo et dispositif électronique
WO2023029916A1 (fr) Procédé et appareil d'affichage d'annotation, dispositif terminal et support de stockage lisible
CN113364964B (zh) 图像处理方法、图像处理装置、存储介质与终端设备
CN115802148A (zh) 一种获取图像的方法及电子设备

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21768492

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21768492

Country of ref document: EP

Kind code of ref document: A1