CN114245129A - Image processing method, image processing device, computer equipment and storage medium - Google Patents
Image processing method, image processing device, computer equipment and storage medium Download PDFInfo
- Publication number
- CN114245129A CN114245129A CN202210162523.0A CN202210162523A CN114245129A CN 114245129 A CN114245129 A CN 114245129A CN 202210162523 A CN202210162523 A CN 202210162523A CN 114245129 A CN114245129 A CN 114245129A
- Authority
- CN
- China
- Prior art keywords
- image
- memory
- video
- path
- data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
- H04N19/136—Incoming video signal characteristics or properties
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/42—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation
- H04N19/423—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation characterised by memory arrangements
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/42—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation
- H04N19/436—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation using parallelised computational arrangements
Abstract
The application discloses an image processing method, an image processing device, computer equipment and a storage medium, wherein the image processing method comprises the following steps: acquiring a plurality of paths of video images and preset target resolution, wherein the target resolution is the image resolution supported by an encoder; determining first memory image information corresponding to each path of video image and a first memory first address corresponding to image data of each path of video image according to the target resolution and the image format of each path of video image in the multiple paths of video images; and respectively writing the image data of each path of video image into the memory according to the first memory head address and the first memory image information. The method and the device can splice multiple paths of video images into spliced video images which can be coded by the coder, so that the multiple paths of video images can be coded by one coder.
Description
Technical Field
The present application relates to the field of communications technologies, and in particular, to an image processing method and apparatus, a computer device, and a storage medium.
Background
With the rapid development of multimedia technology, videos have entered the high definition era, and the application range of the videos is wider and wider. The video is a continuous image sequence and is composed of continuous frames, and because the similarity between the continuous frames is extremely high, in order to facilitate storage and transmission, the original video needs to be encoded and compressed to remove redundancy in spatial and temporal dimensions, and the encoded video data needs to be decoded when the video is watched. The existing chip with one encoder can only process one path of video images generally, and multiple paths of video images cannot be encoded through one encoder.
Disclosure of Invention
The embodiment of the application provides an image processing method, an image processing device, computer equipment and a storage medium, which can splice multiple paths of video images into spliced video images which can be coded by a coder, so that the multiple paths of video images can be coded by one coder.
In one aspect, the present application provides an image processing method applied to an image processing apparatus including an encoder, the image processing method including:
acquiring a plurality of paths of video images and preset target resolution, wherein the target resolution is the image resolution supported by the encoder;
determining first memory image information corresponding to each path of video image and a first memory initial address corresponding to image data of each path of video image according to the target resolution and the image format of each path of video image in the multi-path video images;
and respectively writing the image data of each path of video image into a memory according to the first memory head address and the first memory image information so as to splice the multiple paths of video images into spliced video images which can be coded by the coder.
In some embodiments of the present application, the determining, according to the target resolution and an image format of each of the multiple channels of video images, first memory image information corresponding to each of the multiple channels of video images and a first memory first address corresponding to image data of each of the multiple channels of video images includes:
determining first memory image information corresponding to each path of video image according to the target resolution and the image format of each path of video image in the multi-path video images;
and determining a first memory first address corresponding to the image data of each path of video image according to the image format and the first memory image information.
In some embodiments of the present application, the first memory image information comprises a memory image size and a memory image line span;
determining first memory image information corresponding to each path of video image according to the target resolution and the image format of each path of video image in the multiple paths of video images, wherein the determining comprises the following steps:
determining the size of the memory image corresponding to each path of video image according to the target resolution;
and determining the line span of the memory image corresponding to each path of video image according to the image format of each path of video image in the multi-path video images and the size of the memory image.
In some embodiments of the present application, the memory image size comprises a memory image width and a memory image height;
determining the line span of the memory image corresponding to each path of video image according to the image format of each path of video image in the multiple paths of video images and the size of the memory image, wherein the determining comprises the following steps:
determining the number of pixel bytes corresponding to each path of video image according to the image format of each path of video image in the multiple paths of video images, wherein the number of the pixel bytes is the number of bytes corresponding to a single pixel in each path of video image;
and determining the line span of the memory image corresponding to each path of video image according to the number of bytes of the pixel and the width of the memory image.
In some embodiments of the present application, the memory image line span is N times the product of the image width and the number of pixel bytes, N is greater than or equal to 2, and N is an integer.
In some embodiments of the present application, the determining a first memory head address corresponding to image data of each video image according to the image format and the first memory image information includes:
determining the data type corresponding to the image data of each path of video image and the data amount corresponding to each type of image data according to the image format;
and determining a first memory first address corresponding to each type of image data of each path of video image according to the first memory image information and the data volume.
In some embodiments of the present application, after the respectively writing the image data of the video images into a memory according to the first memory first address and the first memory image information, the method further includes:
encoding image data of the spliced video image to obtain encoded data corresponding to the spliced video image;
and decoding the coded data to obtain decoded data corresponding to the spliced video image.
In some embodiments of the present application, after the decoding the encoded data to obtain the decoded data corresponding to the spliced video image, the method further includes:
acquiring a display request of the multiple paths of video images, and determining second memory image information corresponding to the video images to be displayed and second memory initial addresses corresponding to image data of the video images to be displayed according to the display request;
and reading the decoded data according to the second memory image information and the second memory first address so as to realize the display of the video image to be displayed.
In another aspect, the present application provides an image processing apparatus comprising: the device comprises an image acquisition unit, an information determination unit, an image splicing unit and an encoder;
the image acquisition unit is used for acquiring a plurality of paths of video images and preset target resolution, and the target resolution is the image resolution supported by the encoder;
the information determining unit is used for determining first memory image information corresponding to each path of video image and a first memory first address corresponding to image data of each path of video image according to the target resolution and the image format of each path of video image in the multiple paths of video images;
the image splicing unit is used for respectively writing the image data of each path of video image into a memory according to the first memory initial address and the first memory image information so as to splice the multiple paths of video images into spliced video images which can be coded by the coder.
In another aspect, the present application further provides a computer device, including:
one or more processors;
a memory; and
one or more application programs, wherein the one or more application programs are stored in the memory and configured to be executed by the processor to implement the image processing method of any of the first aspects.
In a fourth aspect, the present application further provides a computer-readable storage medium, on which a computer program is stored, the computer program being loaded by a processor to perform the steps in the image processing method according to any one of the first aspect.
According to the image processing method, the image processing device, the computer equipment and the storage medium, the image data of each path of video image are respectively written into the memories according to the image resolution supported by the encoder and the image format of each path of video image in the multiple paths of video images, the multiple paths of video images can be spliced into spliced video images which can be encoded by the encoder, and the multiple paths of video images can be encoded by one encoder.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.
FIG. 1 is a schematic view of a scene of an image processing system provided in an embodiment of the present application;
FIG. 2 is a flow chart illustrating an embodiment of an image processing method provided in an embodiment of the present application;
FIG. 3 is a flowchart illustrating an exemplary embodiment of an image processing method provided in an embodiment of the present application;
fig. 4 is a graph of the stitching effect of 4 video images written into the memory when the target resolution is 3840 × 2160 in the embodiment of the present application;
fig. 5 is a graph of the stitching effect of 9 video images written into the memory when the target resolution is 3840 × 2160 in the embodiment of the present application;
FIG. 6 is a schematic structural diagram of an embodiment of an image processing apparatus provided in an embodiment of the present application;
fig. 7 is a schematic structural diagram of an embodiment of a computer device provided in an embodiment of the present application.
Detailed Description
The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
In the description of the present application, it is to be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", and the like indicate orientations or positional relationships based on those shown in the drawings, and are used merely for convenience of description and for simplicity of description, and do not indicate or imply that the referenced device or element must have a particular orientation, be constructed in a particular orientation, and be operated, and thus should not be considered as limiting the present application. Furthermore, the terms "first", "second" and "first" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, features defined as "first", "second", may explicitly or implicitly include one or more of the described features. In the description of the present application, "a plurality" means two or more unless specifically limited otherwise.
In this application, the word "exemplary" is used to mean "serving as an example, instance, or illustration. Any embodiment described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other embodiments. The following description is presented to enable any person skilled in the art to make and use the application. In the following description, details are set forth for the purpose of explanation. It will be apparent to one of ordinary skill in the art that the present application may be practiced without these specific details. In other instances, well-known structures and processes are not set forth in detail in order to avoid obscuring the description of the present application with unnecessary detail. Thus, the present application is not intended to be limited to the embodiments shown, but is to be accorded the widest scope consistent with the principles and features disclosed herein.
It should be noted that, since the method in the embodiment of the present application is executed in a computer device, processing objects of each computer device all exist in the form of data or information, for example, time, which is substantially time information, and it is understood that, in the subsequent embodiments, if size, number, position, and the like are mentioned, corresponding data exist so as to be processed by the computer device, and details are not described herein.
Embodiments of the present application provide an image processing method, an image processing apparatus, a computer device, and a storage medium, which are described in detail below.
Referring to fig. 1, fig. 1 is a schematic view of a scene of an image processing system according to an embodiment of the present disclosure, where the image processing system may include a camera module 100, an image processing apparatus 200, and a display screen 300. The camera module 100 includes a plurality of cameras, for example, the camera module 100 includes four cameras, which are a camera 0, a camera 1, a camera 2, and a camera 3, respectively. When the image Processing system is an on-board system, the image Processing apparatus 200 may be an image Processing module in an on-board chip, and the image Processing apparatus 200 includes an image Processing Unit, a DMA (Direct Memory Access) controller, a CPU (Central Processing Unit), an encoder, a decoder, an image display Unit, a bus, a DDR (Double Data Rate) controller, a DDR (Secure Digital Memory) card controller, and an SD card. For example, when the plurality of cameras in the camera module 100 are respectively a camera 0, a camera 1, a camera 2, and a camera 3, the image processing unit includes an image processing unit 0, an image processing unit 1, an image processing unit 2, and an image processing unit 3 connected to the camera 0, the camera 1, the camera 2, and the camera 3, respectively, and the DMA controller includes a DMA controller 0, a DMA controller 1, a DMA controller 2, and a DMA controller 3 connected to the image processing unit 0, the image processing unit 1, the image processing unit 2, and the image processing unit 3, respectively.
In the embodiment of the present application, the image processing apparatus 200 is mainly configured to obtain multiple paths of video images and a predetermined target resolution, where the target resolution is an image resolution supported by an encoder in the image processing apparatus 200, determine, according to the target resolution and an image format of each path of video image in the multiple paths of video images, first memory image information corresponding to each path of video image and a first memory head address corresponding to image data of each path of video image, then write image data of each path of video image into a DDR respectively by using a multiple path DMA controller according to the first memory head address and the first memory image information, after the multiple paths of video images are written, the multiple path DMA controller respectively sends an interrupt to tell a CPU that the image frame has been written, and the CPU starts the encoder to encode image data of a spliced video image, and the coded data obtained after coding is stored in the SD card. When a video image collected by the camera module 100 needs to be watched, the encoded data stored in the SD card can be decoded by the decoder, and the decoded data obtained by decoding is transmitted to the display screen 300 by the image display module for display.
Those skilled in the art will understand that the application environment shown in fig. 1 is only one application scenario of the present application, and does not constitute a limitation on the application scenario of the present application, and that other application environments may further include more or less modules than those shown in fig. 1, for example, only 4 cameras are shown in fig. 1, the camera module 100 may further include more or less cameras, such as 3 cameras or 5 cameras, and the number of the image processing units and the DMA controllers in the corresponding image processing apparatus 200 may also vary, and it is understood that the image processing system may further include one or more other services, which is not limited herein.
It should be noted that the scene schematic diagram of the image processing system shown in fig. 1 is only an example, and the image processing system and the scene described in the embodiment of the present application are for more clearly illustrating the technical solution of the embodiment of the present application, and do not constitute a limitation to the technical solution provided in the embodiment of the present application, and as a person having ordinary skill in the art knows that along with the evolution of the image processing system and the appearance of a new service scene, the technical solution provided in the embodiment of the present application is also applicable to similar technical problems.
First, an embodiment of the present application provides an image processing method, an execution subject of the image processing method being an image processing apparatus applied to a computer device, the image processing apparatus including an encoder, the image processing method including: acquiring a plurality of paths of video images and preset target resolution, wherein the target resolution is the image resolution supported by the encoder; determining first memory image information corresponding to each path of video image and a first memory initial address corresponding to image data of each path of video image according to the target resolution and the image format of each path of video image in the multi-path video images; and respectively writing the image data of each path of video image into a memory according to the first memory head address and the first memory image information so as to splice the multiple paths of video images into spliced video images which can be coded by the coder.
As shown in fig. 2, which is a schematic flowchart of an embodiment of an image processing method in an embodiment of the present application, the image processing method includes:
s100, obtaining a plurality of paths of video images and preset target resolution, wherein the target resolution is the image resolution supported by the encoder.
The multi-path video images are video images respectively acquired by a plurality of cameras, for example, the multi-path video images are video images respectively acquired by the camera 0, the camera 1, the camera 2 and the camera 3 in fig. 1. The head portrait processing method in the present embodiment is applied to an image processing apparatus including an encoder, and the target resolution is an image resolution supported by the encoder in the image processing apparatus, for example, the target resolution is an image resolution supported by the encoder in fig. 1, and when the maximum image resolution supported by the encoder is 3840 × 2160, the target resolution may be any resolution less than or equal to 3840 × 2160.
In order to enable the encoder to encode multiple video images simultaneously, the video images after splicing of the multiple video images cannot exceed the image resolution supported by the encoder. Therefore, in this embodiment, the multiple video images acquired by the multiple cameras and the target resolution, that is, the image resolution supported by the encoder, are first obtained, so that the multiple video images are processed according to the target resolution in the subsequent step, and the multiple video images are spliced into a spliced video image that can be encoded by the encoder. To optimize the resolution of the stitched video image, in one embodiment, the maximum image resolution supported by the encoder is determined to be the target resolution, e.g., 3840x2160 when the maximum image resolution supported by the encoder is 3840x 2160.
S200, determining first memory image information corresponding to each path of video image and a first memory first address corresponding to image data of each path of video image according to the target resolution and the image format of each path of video image in the multiple paths of video images.
The image format is a format of a computer storage image, and common image formats include a gray scale image, an RGB format and a YUV format, wherein the RGB format is further classified into RGB565, RGB555, RGB24, RGB888 and the like, and the YUV format is further classified into YUV444, YUV422, YUV420SP, YUV420 and the like. The image data is image data in each path of video image, the video images of different image formats have different types of image data, for example, for the video image of YUV420SP format, there are two types of image data, namely Y component image data and UV component image data; and for a video image in YUV420P format, there are three types of image data, i.e., Y-component image data, U-component image data, and V-component image data.
The memory image information is image information corresponding to each path of video image after being written into the storage, the memory image information of each path of video image includes a memory image size and a memory image line span, the memory image size is an image resolution of each path of video image after being written into the storage, and includes a memory image width and a memory image height, for example, the memory image size is 1920x1080, which means that the memory image width of the video image written into the storage is 1920, and the memory image height is 1080. When a video Image is stored in memory, the end of each line of the Image may contain some extended content that only affects how the Image is stored in memory, but does not affect how the Image is displayed, and the memory Image line span (Image Stride) is the term for these extended content, and the memory Image line span is also called Pitch, and if the end of each line of pixels of the Image has the extended content, the value of the memory Image line span must be greater than the width of the Image.
The memory first address is the first address of the image data in each path of video image in the memory, and the image data of each path of video image is written into the memory according to the memory first address. In consideration of the fact that video images with different image formats have different types of image data, after multiple paths of video images and a predetermined target resolution are obtained, according to the target resolution and the image format of each path of video image in the multiple paths of video images, first memory image information corresponding to each path of video image and a first memory head address corresponding to the image data of each path of video image are determined, so that the video images are stored in a storage according to the first memory head address and the first memory image information in the subsequent steps.
In one embodiment, as shown in fig. 3, step S200 includes:
s210, determining first memory image information corresponding to each path of video image according to the target resolution and the image format of each path of video image in the multiple paths of video images;
s220, determining a first memory first address corresponding to the image data of each path of video image according to the image format and the first memory image information.
In this embodiment, when determining the first memory image information and the first memory first address according to the target resolution and the image format of each of the multiple paths of video images, first determining the first memory image information corresponding to each of the multiple paths of video images according to the target resolution and the image format of each of the multiple paths of video images, and then determining the first memory first address corresponding to the image data of each of the multiple paths of video images according to the image format and the first memory image information.
In one embodiment, step S210 includes:
s211, determining the memory image size corresponding to each path of video image according to the target resolution;
s212, determining the line span of the memory image corresponding to each path of video image according to the image format of each path of video image in the multi-path video images and the size of the memory image.
In the foregoing step, the memory image information includes a memory image size and a memory image line span, that is, the first memory image information includes a memory image size and a memory image line span, and when the first memory image information is determined according to the target resolution and the image format of each video image in the multiple video images, the memory image size of each video image in the multiple video images is determined according to the target resolution. For example, as shown in fig. 4, the multiple paths of video images include 4 paths of video images, the target resolution is 3840x2160, the memory image size of each path of video image is 1920x1080, and the image data of the 4 paths of video images are written into the memory to form a complete path of image 3840x 2160; as shown in fig. 5, the multiple paths of video images include 9 paths of video images, and the target resolution is still 3840x2160, the sizes of the memory images of the respective paths of video images are 1280 x720, and the image data of the 9 paths of video images are written into the memory to form a complete path of image 3840x 2160.
The memory image line span represents the distance between the first address of one image line and the first address of the next line, and generally the size is the image width multiplied by the number of bytes occupied by each pixel, and the image format of each path of video image not only affects the image data type of each path of video image, but also affects the number of bytes occupied by each pixel in each path of video image. Therefore, in this embodiment, after the memory image size of each video image is determined, the line span of the memory image corresponding to each video image in the multiple video images is further determined according to the image format and the memory image width of each video image.
In one embodiment, as shown in fig. 3, step S212 includes:
s2121, determining the number of pixel bytes corresponding to each path of video image according to the image format of each path of video image in the multiple paths of video images, wherein the number of the pixel bytes is the number of bytes corresponding to a single pixel in each path of video image;
and S2122, determining the line span of the memory image corresponding to each path of video image according to the number of bytes of the pixel and the width of the memory image.
The number of bytes of a pixel is the number of bytes corresponding to a single pixel in each path of video image, and the number of bytes corresponding to a single pixel in video images with different image formats is different, for example, for a video image with YUV444 format, one pixel occupies 3 bytes, and for a video image with YUV422 format, one pixel occupies 2 bytes. After the size of the memory image corresponding to each path of video image is determined, the number of bytes of pixels of each path of video image in the multiple paths of video images is determined according to the image format of each path of video image, and then the line span of the memory image of each path of video image is determined according to the number of bytes of pixels and the width of the memory image in the first memory image information. For example, the image format is YUV420SP, and the image size is 1920 × 1080, the memory image line span of each video image may be set to 1920 × 1 or any value greater than 1920 × 1.
In order to skip a plurality of images, the memory image line span is N times of the product of the image width and the number of pixel bytes, wherein N is greater than or equal to 2 and is an integer. For example, when the image format is YUV420SP and the memory image size is 1920 × 1080, the memory image line span of each video image may be set to any value of 1920 × 1 × 2 and 1920 × 1 × 3 … 1920 × 1 × N, where 1 represents the number of pixel bytes corresponding to the video image in YUV420SP format, and N represents skipping N-1 sub-images. In one embodiment, the memory image line span is typically set to 2 to 3 times the product of the image width and the number of pixel bytes, i.e., N is 2 or 3. For example, the image format is YUV420SP, each video image is spliced according to the method shown in fig. 4, and the memory image line span of each video image is set to 1920 × 1 × 2.
In one embodiment, with continued reference to fig. 3, step S220 includes:
s221, determining the data type corresponding to the image data of each path of video image and the data amount corresponding to each type of image data according to the image format;
s222, determining a first memory first address corresponding to each type of image data of each path of video image according to the first memory image information and the data volume.
Video images of different image formats correspond to different types of image data, and the data amount of the different types of image data may be different, for example, for a video image of YUV420SP format, including Y component image data and UV component image data, for the Y component image data, one pixel corresponds to one byte, and two lines of Y component image data share one line of UV component image data, that is, the UV component image data is half of the Y component image data. In this embodiment, when determining the first memory first address, first, according to the image format, the data type corresponding to the image data of each path of video image and the data amount corresponding to each type of image data are determined, for example, for a video image in YUV420SP format, the data type corresponding to the image data of the video image includes Y component image data and UV component image data, and the UV component image data is half of the Y component image data.
After determining the data type corresponding to the image data of each path of video image and the data amount corresponding to each type of image data, determining a first memory head address corresponding to the image data of each path of video image according to the first memory image information and the data amount, wherein the first memory head address corresponding to the image data of each path of video image is a first memory head address corresponding to each type of image data of each path of video image. For example, when the image format of the multiple paths of video images is YUV420SP, since the multiple paths of video images include Y component image data and UV component image data, when determining the first memory head address corresponding to the image data of each path of video images, it is necessary to determine the first memory head address corresponding to the Y component image data and the UV component image data, respectively. As shown in fig. 4, when the image format of each video image is YUV420SP, assuming that the first memory header address corresponding to the Y component image data of the first video image is set to Y _ addr, the first memory header address corresponding to the Y component image data of the second video image is set to Y _ addr +1920, the first memory header address corresponding to the Y component image data of the third video image is set to Y _ addr +1920 x1080 × 2, and the first memory header address corresponding to the Y component image data of the fourth video image is set to Y _ addr +1920 x1080 × 2+1920, where 2 indicates that two images are skipped; assuming that the first memory head address corresponding to the UV component image data of the first path of video image is set to UV _ addr, since the data volume of the UV component image data is half of the data volume of the Y component image data, the first memory head address of the UV component image data of the second path of video image is set to UV _ addr +1920, the first memory head address of the UV component image data of the third path of video image is set to UV _ addr +1920 x1080 × 2/2, and the first memory head address of the UV component image data of the fourth path of video image is set to UV _ addr +1920 x1080 × 2/2+1920, where x2 indicates that two images are skipped, and/2 indicates that the data volume of the UV component image data is half of the data volume of the Y component image data.
S300, respectively writing the image data of each path of video image into a memory according to the first memory initial address and the first memory image information so as to splice the multiple paths of video images into spliced video images which can be coded by the coder.
For example, in the case of a video image in YUV420SP format, the image data of the video image includes Y-component image data and UV-component image data, and in the case of writing the image data of each video image in the memory, the Y-component image data and the UV-component image data are written in the memory. In this embodiment, after determining the first memory image information of each path of video image and the first memory first address corresponding to each type of image data according to the image format and the target resolution, the image data of each path of video image is respectively written into the memory according to the first memory image information of each path of video image and the first memory first address corresponding to each type of image data, so that multiple paths of video images are spliced into a spliced video image that can be encoded by one encoder. For example, the memory may be a DDR, and the DMA controllers respectively write the image data of each video image into the DDR according to the first memory image information corresponding to each video image and the first memory first address corresponding to the image data of each video image. Because each path of video collected by the camera comprises a plurality of frames of video images, after the DMA writes the image data of one frame of video image into the DDR, the image data of the next frame of video image can be continuously written into the DDR according to the first memory image information corresponding to the next frame of video image and the first memory first address corresponding to the image data of the next frame of video image. In a specific embodiment, with reference to fig. 3, after the step S300 of writing the image data of each video image into a memory according to the first memory first address and the first memory image information respectively, the method includes:
s410, encoding image data of the spliced video image to obtain encoded data corresponding to the spliced video image;
and S420, decoding the coded data to obtain decoded data corresponding to the spliced video image.
After the plurality of paths of video images are spliced into the spliced video image, the image data of the spliced video image can be encoded to obtain encoded data corresponding to the spliced video image. When a plurality of paths of video images need to be watched, the coded data can be decoded to obtain decoded data corresponding to the spliced video images, and then the decoded data is transmitted to a display screen for displaying. For example, when the image data of each video image is written into the memory by the multiple DMA controllers, each DMA controller will respectively send an interrupt to the CPU to tell the CPU that the frame of image data has been written, the CPU will start the encoder to encode the stitched image data to obtain encoded data corresponding to the multiple video images, and store the encoded data into the SD card. When the multiple paths of video images need to be watched, the encoded data in the SD card can be read through the decoder, the encoded data are decoded to obtain decoded data corresponding to the multiple paths of video images, and then the decoded data are read by the image display unit and transmitted to the display screen for displaying.
In a specific embodiment, with reference to fig. 3 again, after the decoding the encoded data in step S420 to obtain the decoded data corresponding to the spliced video image, the method includes:
s430, acquiring a display request of the multiple paths of video images, and determining second memory image information corresponding to the video images to be displayed and second memory initial addresses corresponding to image data of the video images to be displayed according to the display request;
s440, reading the decoded data according to the second memory image information and the second memory first address to realize the display of the video image to be displayed.
The image processing device 200 may obtain the display request of the multiple channels of video images sent by the user, determine second memory image information corresponding to the video images to be displayed and second memory head addresses corresponding to the image data of the video images to be displayed according to the display request, and then read decoded data according to the second memory image information and the second memory head addresses to display the video images to be displayed, wherein the second memory image information corresponding to the video images to be displayed includes memory image sizes and memory image line spans of the video images to be displayed. In this embodiment, the decoding data is read according to the second memory image information and the second memory first address, so that all or part of the multiple paths of video images can be displayed. For example, as shown in fig. 4, when it is determined that the first channel of video image needs to be displayed according to the display request, the image display unit reads the decoded data according to the memory image information and the memory first address corresponding to the first channel of video image, and transmits the read decoded data to the display screen for display.
In order to better implement the image processing method in the embodiment of the present application, on the basis of the image processing method, an image processing apparatus is further provided in the embodiment of the present application, as shown in fig. 6, the image processing apparatus 600 includes: an image acquisition unit 601, an information determination unit 602, an image stitching unit 603, and an encoder.
The image obtaining unit 601 is configured to obtain multiple channels of video images and a preset target resolution, where the target resolution is an image resolution supported by the encoder;
the information determining unit 602 is configured to determine, according to the target resolution and an image format of each of the multiple paths of video images, first memory image information corresponding to each of the multiple paths of video images and a first memory first address corresponding to image data of each of the multiple paths of video images;
the image stitching unit 603 is configured to write the image data of each video image into a memory according to the first memory first address and the first memory image information, so as to stitch the multiple video images into a stitched video image that can be encoded by the encoder;
the encoder is used for encoding the image data of the spliced video image to obtain the encoded data corresponding to the spliced video image.
In the embodiment of the application, the image data of each path of video image is respectively written into the memory according to the image resolution supported by the encoder and the image formats of the multiple paths of video images, and the multiple paths of video images can be spliced into the spliced video image which can be encoded by the encoder, so that the multiple paths of video images can be encoded by one encoder.
In some embodiments of the present application, the information determining unit 602 is specifically configured to:
determining first memory image information corresponding to each path of video image according to the target resolution and the image format of each path of video image in the multi-path video images;
and determining a first memory first address corresponding to the image data of each path of video image according to the image format and the first memory image information.
In some embodiments of the present application, the information determining unit 602 is further specifically configured to:
determining the size of the memory image corresponding to each path of video image according to the target resolution;
and determining the line span of the memory image corresponding to each path of video image according to the image format of each path of video image in the multi-path video images and the size of the memory image.
In some embodiments of the present application, the information determining unit 602 is further specifically configured to:
determining the number of pixel bytes corresponding to each path of video image according to the image format of each path of video image in the multiple paths of video images, wherein the number of the pixel bytes is the number of bytes corresponding to a single pixel in each path of video image;
and determining the line span of the memory image corresponding to each path of video image according to the number of bytes of the pixel and the width of the memory image.
In some embodiments of the present application, the information determining unit 602 is further specifically configured to:
determining the data type corresponding to the image data of each path of video image and the data amount corresponding to each type of image data according to the image format;
and determining a first memory first address corresponding to each type of image data of each path of video image according to the first memory image information and the data volume.
In some embodiments of the present application, the image processing apparatus 600 further comprises:
and the decoder is used for decoding the coded data to obtain the decoded data corresponding to the spliced video image.
In some embodiments of the present application, the image processing apparatus 600 further comprises:
the request acquisition unit is used for acquiring a display request of the multi-channel video images and determining second memory image information corresponding to the video images to be displayed and second memory initial addresses corresponding to the image data of the video images to be displayed according to the display request;
and the data reading unit is used for reading the decoded data according to the second memory image information and the second memory head address so as to realize the display of the video image to be displayed.
An embodiment of the present application further provides a computer device, into which any one of the image processing apparatuses provided in the embodiment of the present application is integrated, where the computer device includes:
one or more processors;
a memory; and
one or more application programs, wherein the one or more application programs are stored in the memory and configured to be executed by the processor for performing the steps of the image processing method as described in any of the above-described embodiments of the image processing method.
The embodiment of the application also provides computer equipment, which integrates any image processing device provided by the embodiment of the application. Fig. 7 is a schematic diagram showing a structure of a computer device according to an embodiment of the present application, specifically:
the computer device may include components such as a processor 701 of one or more processing cores, memory 702 of one or more computer-readable storage media, a power supply 703, and an input unit 704. Those skilled in the art will appreciate that the computer device configuration illustrated in FIG. 7 does not constitute a limitation of computer devices, and may include more or fewer components than those illustrated, or some components may be combined, or a different arrangement of components. Wherein:
the processor 701 is a control center of the computer apparatus, connects various parts of the entire computer apparatus using various interfaces and lines, and performs various functions of the computer apparatus and processes data by running or executing software programs and/or modules stored in the memory 702 and calling data stored in the memory 702, thereby monitoring the computer apparatus as a whole. Optionally, processor 701 may include one or more processing cores; preferably, the processor 701 may integrate an application processor, which mainly handles operating systems, user interfaces, application programs, etc., and a modem processor, which mainly handles wireless communications. It will be appreciated that the modem processor described above may not be integrated into the processor 701.
The memory 702 may be used to store software programs and modules, and the processor 701 executes various functional applications and data processing by operating the software programs and modules stored in the memory 702. The memory 702 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application program required by at least one function (such as a sound playing function, an image playing function, etc.), and the like; the storage data area may store data created according to use of the computer device, and the like. Further, the memory 702 may include high speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid state storage device. Accordingly, the memory 702 may also include a memory controller to provide the processor 701 with access to the memory 702.
The computer device further includes a power supply 703 for supplying power to the various components, and preferably, the power supply 703 is logically connected to the processor 701 through a power management system, so that functions of managing charging, discharging, and power consumption are implemented through the power management system. The power supply 703 may also include any component including one or more of a dc or ac power source, a recharging system, a power failure detection circuit, a power converter or inverter, a power status indicator, and the like.
The computer device may also include an input unit 704, the input unit 704 being operable to receive input numeric or character information and generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function control.
Although not shown, the computer device may further include a display unit and the like, which are not described in detail herein. Specifically, in this embodiment, the processor 701 in the computer device loads the executable file corresponding to the process of one or more application programs into the memory 702 according to the following instructions, and the processor 701 runs the application program stored in the memory 702, thereby implementing various functions as follows:
acquiring a plurality of paths of video images and preset target resolution, wherein the target resolution is the image resolution supported by the encoder;
determining first memory image information corresponding to each path of video image and a first memory initial address corresponding to image data of each path of video image according to the target resolution and the image format of each path of video image in the multi-path video images;
and respectively writing the image data of each path of video image into a memory according to the first memory head address and the first memory image information so as to splice the multiple paths of video images into spliced video images which can be coded by the coder.
It will be understood by those skilled in the art that all or part of the steps of the methods of the above embodiments may be performed by instructions or by associated hardware controlled by the instructions, which may be stored in a computer readable storage medium and loaded and executed by a processor.
To this end, an embodiment of the present application provides a computer-readable storage medium, which may include: read Only Memory (ROM), Random Access Memory (RAM), magnetic or optical disks, and the like. The image processing method comprises a step of storing a computer program, and a step of loading the computer program by a processor to execute the steps of any image processing method provided by the embodiment of the application. For example, the computer program may be loaded by a processor to perform the steps of:
acquiring a plurality of paths of video images and preset target resolution, wherein the target resolution is the image resolution supported by the encoder;
determining first memory image information corresponding to each path of video image and a first memory initial address corresponding to image data of each path of video image according to the target resolution and the image format of each path of video image in the multi-path video images;
and respectively writing the image data of each path of video image into a memory according to the first memory head address and the first memory image information so as to splice the multiple paths of video images into spliced video images which can be coded by the coder.
In the above embodiments, the descriptions of the respective embodiments have respective emphasis, and parts that are not described in detail in a certain embodiment may refer to the above detailed descriptions of other embodiments, and are not described herein again.
In a specific implementation, each unit or structure may be implemented as an independent entity, or may be combined arbitrarily to be implemented as one or several entities, and the specific implementation of each unit or structure may refer to the foregoing method embodiment, which is not described herein again.
The above operations can be implemented in the foregoing embodiments, and are not described in detail herein.
The foregoing detailed description has provided an image processing method, an image processing apparatus, a computer device, and a storage medium according to embodiments of the present application, and specific examples have been applied in the present application to explain the principles and implementations of the present application, and the descriptions of the foregoing embodiments are only used to help understand the method and the core ideas of the present application; meanwhile, for those skilled in the art, according to the idea of the present application, there may be variations in the specific embodiments and the application scope, and in summary, the content of the present specification should not be construed as a limitation to the present application.
Claims (11)
1. An image processing method applied to an image processing apparatus including an encoder, the image processing method comprising:
acquiring a plurality of paths of video images and preset target resolution, wherein the target resolution is the image resolution supported by the encoder;
determining first memory image information corresponding to each path of video image and a first memory initial address corresponding to image data of each path of video image according to the target resolution and the image format of each path of video image in the multi-path video images;
and respectively writing the image data of each path of video image into a memory according to the first memory head address and the first memory image information so as to splice the multiple paths of video images into spliced video images which can be coded by the coder.
2. The image processing method according to claim 1, wherein determining, according to the target resolution and an image format of each of the multiple paths of video images, first memory image information corresponding to each of the paths of video images and a first memory header address corresponding to image data of each of the paths of video images, comprises:
determining first memory image information corresponding to each path of video image according to the target resolution and the image format of each path of video image in the multi-path video images;
and determining a first memory first address corresponding to the image data of each path of video image according to the image format and the first memory image information.
3. The image processing method according to claim 2, wherein the first in-memory image information includes an in-memory image size and an in-memory image line span;
determining first memory image information corresponding to each path of video image according to the target resolution and the image format of each path of video image in the multiple paths of video images, wherein the determining comprises the following steps:
determining the size of the memory image corresponding to each path of video image according to the target resolution;
and determining the line span of the memory image corresponding to each path of video image according to the image format of each path of video image in the multi-path video images and the size of the memory image.
4. The image processing method according to claim 3, wherein the memory image size comprises a memory image width and a memory image height;
determining the line span of the memory image corresponding to each path of video image according to the image format of each path of video image in the multiple paths of video images and the size of the memory image, wherein the determining comprises the following steps:
determining the number of pixel bytes corresponding to each path of video image according to the image format of each path of video image in the multiple paths of video images, wherein the number of the pixel bytes is the number of bytes corresponding to a single pixel in each path of video image;
and determining the line span of the memory image corresponding to each path of video image according to the number of bytes of the pixel and the width of the memory image.
5. The image processing method of claim 4, wherein the memory image line span is N times the product of the image width and the number of pixel bytes, N is greater than or equal to 2, and N is an integer.
6. The image processing method according to claim 2, wherein the determining a first memory head address corresponding to the image data of each video image according to the image format and the first memory image information comprises:
determining the data type corresponding to the image data of each path of video image and the data amount corresponding to each type of image data according to the image format;
and determining a first memory first address corresponding to each type of image data of each path of video image according to the first memory image information and the data volume.
7. The image processing method according to claim 1, wherein after the image data of the video images are written into a memory according to the first memory head address and the first memory image information, the method further comprises:
encoding image data of the spliced video image to obtain encoded data corresponding to the spliced video image;
and decoding the coded data to obtain decoded data corresponding to the spliced video image.
8. The image processing method according to claim 7, wherein after the decoding of the encoded data to obtain the decoded data corresponding to the spliced video image, the method further comprises:
acquiring a display request of the multiple paths of video images, and determining second memory image information corresponding to the video images to be displayed and second memory initial addresses corresponding to image data of the video images to be displayed according to the display request;
and reading the decoded data according to the second memory image information and the second memory first address so as to realize the display of the video image to be displayed.
9. An image processing apparatus characterized by comprising: the device comprises an image acquisition unit, an information determination unit, an image splicing unit and an encoder;
the image acquisition unit is used for acquiring a plurality of paths of video images and preset target resolution, and the target resolution is the image resolution supported by the encoder;
the information determining unit is used for determining first memory image information corresponding to each path of video image and a first memory first address corresponding to image data of each path of video image according to the target resolution and the image format of each path of video image in the multiple paths of video images;
the image splicing unit is used for respectively writing the image data of each path of video image into a memory according to the first memory initial address and the first memory image information so as to splice the multiple paths of video images into spliced video images which can be coded by the coder.
10. A computer device, characterized in that the computer device comprises:
one or more processors;
a memory; and
one or more application programs, wherein the one or more application programs are stored in the memory and configured to be executed by the processor to implement the image processing method of any of claims 1 to 8.
11. A computer-readable storage medium, having stored thereon a computer program which is loaded by a processor for performing the steps of the image processing method of any one of claims 1 to 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210162523.0A CN114245129A (en) | 2022-02-22 | 2022-02-22 | Image processing method, image processing device, computer equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210162523.0A CN114245129A (en) | 2022-02-22 | 2022-02-22 | Image processing method, image processing device, computer equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114245129A true CN114245129A (en) | 2022-03-25 |
Family
ID=80747745
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210162523.0A Pending CN114245129A (en) | 2022-02-22 | 2022-02-22 | Image processing method, image processing device, computer equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114245129A (en) |
Citations (16)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20030085988A1 (en) * | 1998-02-27 | 2003-05-08 | Hideyuki Fujiwara | Image sensing and transfer apparatus and method |
| CN1450807A (en) * | 2002-04-09 | 2003-10-22 | 华为技术有限公司 | Double-vision transmission system for television conference terminal |
| US20050114901A1 (en) * | 2003-10-03 | 2005-05-26 | Canon Kabushiki Kaisha | Information processor, TV system, control method and program |
| CN102801979A (en) * | 2012-08-09 | 2012-11-28 | 武汉微创光电股份有限公司 | Multi-channel video hybrid coding method and device |
| US20130322552A1 (en) * | 2011-12-12 | 2013-12-05 | Animesh Mishra | Capturing Multiple Video Channels for Video Analytics and Encoding |
| US20130342873A1 (en) * | 2012-06-20 | 2013-12-26 | Canon Kabushiki Kaisha | Image processing apparatus, method thereof,and storage medium |
| CN103686047A (en) * | 2013-12-18 | 2014-03-26 | 电子科技大学 | Multi-channel video data transmission method |
| CN104767973A (en) * | 2015-04-08 | 2015-07-08 | 北京航空航天大学 | Multi-channel video real-time wireless transmission and display system and construction method |
| CN205160696U (en) * | 2015-12-10 | 2016-04-13 | 北京国基科技股份有限公司 | Show accuse equipment with multichannel video concatenation function |
| CN105578129A (en) * | 2015-12-14 | 2016-05-11 | 谭焕玲 | Multipath multi-image video splicing device |
| CN107333062A (en) * | 2017-07-05 | 2017-11-07 | 浙江华创视讯科技有限公司 | A kind of method, system and device of Computer Vision |
| CN108616717A (en) * | 2016-12-12 | 2018-10-02 | 中国航空工业集团公司西安航空计算技术研究所 | A kind of real-time panoramic video splicing display device and its method |
| CN109218656A (en) * | 2017-06-30 | 2019-01-15 | 杭州海康威视数字技术股份有限公司 | Image display method, apparatus and system |
| CN112188136A (en) * | 2020-09-24 | 2021-01-05 | 高新兴科技集团股份有限公司 | Method, system, storage medium and equipment for splicing and recording videos in real time in all-in-one mode |
| CN112422868A (en) * | 2020-10-14 | 2021-02-26 | 西安万像电子科技有限公司 | Data processing method, terminal device and server |
| WO2021179783A1 (en) * | 2020-03-11 | 2021-09-16 | 叠境数字科技（上海）有限公司 | Free viewpoint-based video live broadcast processing method, device, system, chip and medium |
-
2022
- 2022-02-22 CN CN202210162523.0A patent/CN114245129A/en active Pending
Patent Citations (16)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20030085988A1 (en) * | 1998-02-27 | 2003-05-08 | Hideyuki Fujiwara | Image sensing and transfer apparatus and method |
| CN1450807A (en) * | 2002-04-09 | 2003-10-22 | 华为技术有限公司 | Double-vision transmission system for television conference terminal |
| US20050114901A1 (en) * | 2003-10-03 | 2005-05-26 | Canon Kabushiki Kaisha | Information processor, TV system, control method and program |
| US20130322552A1 (en) * | 2011-12-12 | 2013-12-05 | Animesh Mishra | Capturing Multiple Video Channels for Video Analytics and Encoding |
| US20130342873A1 (en) * | 2012-06-20 | 2013-12-26 | Canon Kabushiki Kaisha | Image processing apparatus, method thereof,and storage medium |
| CN102801979A (en) * | 2012-08-09 | 2012-11-28 | 武汉微创光电股份有限公司 | Multi-channel video hybrid coding method and device |
| CN103686047A (en) * | 2013-12-18 | 2014-03-26 | 电子科技大学 | Multi-channel video data transmission method |
| CN104767973A (en) * | 2015-04-08 | 2015-07-08 | 北京航空航天大学 | Multi-channel video real-time wireless transmission and display system and construction method |
| CN205160696U (en) * | 2015-12-10 | 2016-04-13 | 北京国基科技股份有限公司 | Show accuse equipment with multichannel video concatenation function |
| CN105578129A (en) * | 2015-12-14 | 2016-05-11 | 谭焕玲 | Multipath multi-image video splicing device |
| CN108616717A (en) * | 2016-12-12 | 2018-10-02 | 中国航空工业集团公司西安航空计算技术研究所 | A kind of real-time panoramic video splicing display device and its method |
| CN109218656A (en) * | 2017-06-30 | 2019-01-15 | 杭州海康威视数字技术股份有限公司 | Image display method, apparatus and system |
| CN107333062A (en) * | 2017-07-05 | 2017-11-07 | 浙江华创视讯科技有限公司 | A kind of method, system and device of Computer Vision |
| WO2021179783A1 (en) * | 2020-03-11 | 2021-09-16 | 叠境数字科技（上海）有限公司 | Free viewpoint-based video live broadcast processing method, device, system, chip and medium |
| CN112188136A (en) * | 2020-09-24 | 2021-01-05 | 高新兴科技集团股份有限公司 | Method, system, storage medium and equipment for splicing and recording videos in real time in all-in-one mode |
| CN112422868A (en) * | 2020-10-14 | 2021-02-26 | 西安万像电子科技有限公司 | Data processing method, terminal device and server |
Non-Patent Citations (2)
| Title |
|---|
| 俞柏峰等: "多画面视频图像分割拼接实现方法", 《长江大学学报(自然科学版)理工卷》 * |
| 俞柏峰等: "多画面视频图像分割拼接实现方法", 《长江大学学报(自然科学版)理工卷》, no. 03, 15 September 2009 (2009-09-15) * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP4789753B2 (en) | Image data buffer device, image transfer processing system, and image data buffer method | |
| US9307244B2 (en) | Data storage controlling device, data reading controlling device, and data storing method | |
| JP3162773B2 (en) | Image processing method and apparatus | |
| JP2007189518A (en) | Signal processing apparatus, imaging device, network camera system, and video system | |
| US20080259089A1 (en) | Apparatus and method for performing motion compensation by macro block unit while decoding compressed motion picture | |
| CN115209145A (en) | Video compression method, system, device and readable storage medium | |
| CN115297094A (en) | Video transmission control method, device, equipment and computer readable storage medium | |
| EP2166770B1 (en) | Moving-image reproducing apparatus and moving-image reproducing method | |
| JP4195969B2 (en) | Motion vector detection device | |
| JPH09205651A (en) | Image data decoding method and decoder using same | |
| KR102576630B1 (en) | An operation method of a decoder, and an operation method of an application processor including the decoder | |
| US9627005B2 (en) | Video processing apparatus and control method of video processing apparatus | |
| CN111405347B (en) | Picture display method and device, electronic equipment and readable storage medium | |
| CN110446046B (en) | Batch image fast decoding method based on embedded platform | |
| CN114245129A (en) | Image processing method, image processing device, computer equipment and storage medium | |
| US8737467B2 (en) | Information processing apparatus and method | |
| KR19980018884A (en) | An image processor (Image Processor) | |
| CN116880937A (en) | Desktop screen capturing data processing method, device, equipment and medium for interactive classroom | |
| CN112911390B (en) | Video data playing method and terminal equipment | |
| TWI603616B (en) | On die/off die memory management | |
| US20060170708A1 (en) | Circuits for processing encoded image data using reduced external memory access and methods of operating the same | |
| US7457470B2 (en) | Image processing apparatus | |
| US8264496B2 (en) | Data management for image processing | |
| CN116166830B (en) | Data processing system, electronic component, electronic device and data processing method | |
| US20110026603A1 (en) | Method and apparatus of using system memory for 3d comb filtering for video decoding |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20220325 |
|
| RJ01 | Rejection of invention patent application after publication |