WO2022205157A1

WO2022205157A1 - Image transmission method and device

Info

Publication number: WO2022205157A1
Application number: PCT/CN2021/084648
Authority: WO
Inventors: 黄汉林
Original assignee: 西门子（中国）有限公司
Priority date: 2021-03-31
Filing date: 2021-03-31
Publication date: 2022-10-06
Also published as: CN117063472A

Abstract

The present invention provides an image transmission method. The image transmission method comprises: decoding a received image stream to obtain a plurality of decoded images and image parameters corresponding to each of the plurality of decoded images, the image parameters comprising the length, height, and number of channels of the decoded image; generating a header message of the decoded image according to the image parameters, and integrating the decoded image and the header message of the decoded image and packaging same into a single image data packet; sending the image data packets to a plurality of image processing modules that are independent of each other, and parsing and processing the received image data packets by the image processing modules.

Description

An image transmission method and device

technical field

The present invention mainly relates to the field of images, and in particular, to an image transmission method and device.

Background technique

The image is usually stored on the file server in an encoded form. After the image processing module obtains the image from the file server, it needs to decode the image before performing the image processing function. In the prior art, the image processing module and the decoding module are coupled, that is, one image processing module will be set with a corresponding decoding module, the input of the decoding module is used as the input of the corresponding image processing module, and each image processing module is obtained from the file server. After the image, the image needs to be decoded. This architecture makes the operation of the entire system very complicated, and the hardware requirements are also extremely strict.

SUMMARY OF THE INVENTION

In order to solve the above technical problems, the present invention provides an image transmission method and device to reduce the computational complexity of the system and the requirements for hardware.

In order to achieve the above object, the present invention provides an image transmission method, the image transmission method includes: decoding a received image stream to obtain a plurality of decoded images and an image corresponding to each decoded image in the plurality of decoded images parameters, the image parameters include the length, height and number of channels of the decoded image, and the image stream includes multiple encoded images; the header message of each decoded image is generated according to the image parameter, and the decoded image The image and the header message of each decoded image are integrated and packaged into a single image data packet; the image data packet is sent to a plurality of mutually independent image processing modules, and each of the image processing modules parses and processes the received image data Bag. For this reason, by encapsulating the decoded image and the header message of the decoded image into image data packets and sending them to multiple independent image processing modules, only one decoding is needed for multiple independent image processing modules to perform image processing, reducing the need for image processing. The computational complexity of the system and the requirements for hardware.

In an embodiment of the present invention, integrating and encapsulating the respective decoded images and the header messages of the respective decoded images into a single image data packet includes: inserting the header message into the header of the corresponding decoded image, so as to encapsulate all the decoded images into a single image data packet. The respective decoded images and the header messages of the respective decoded images are integrated and packaged into a single image data packet. Therefore, by inserting the header message into the header of the corresponding decoded image, the original structure of the decoded image does not need to be adjusted, and the generation speed of the image data packet is improved.

In an embodiment of the present invention, the image parameter further includes an image sequence number, and the method further includes: inserting the header message into the header of a corresponding decoded image, and arranging the decoded images with the header message according to the image sequence number image. Therefore, arranging the decoded images with the header message according to the image sequence number can improve the order of the decoded images, thereby improving the efficiency of image processing.

In an embodiment of the present invention, the header message further includes a start character and an end character, and each of the image processing modules parses and processes the received image data packet includes: determining the received image data packet according to the start character and the end character. position of the header message, and the position of the decoded image is determined according to the length, height and number of channels. To this end, the localization of header messages and decoded images can be achieved.

In an embodiment of the present invention, the method includes: using a publish/subscribe message mode to send the image data packet to a plurality of mutually independent image processing modules. To this end, by using the publish/subscriber message mode to send image data packets to multiple independent image processing modules, it can ensure that the image data packets are sent to the image processing modules to prevent packet loss.

An image transmission device, the image transmission device comprising: a decoding module for decoding a received image stream to obtain multiple decoded images and image parameters corresponding to each decoded image in the multiple decoded images, the image parameters Including the length, height and number of channels of the decoded image, and the image stream includes a plurality of encoded images; the encapsulation module generates a header message of each decoded image according to the image parameter, and combines the decoded image and the The header message of each decoded image is integrated and encapsulated into a single image data packet; the sending module sends the image data packet to a plurality of mutually independent image processing modules, and each of the image processing modules parses and processes the received image data Bag.

In an embodiment of the present invention, the encapsulation module integrating and encapsulating the respective decoded images and the header messages of the respective decoded images into a single image data packet includes: inserting the header message into the header of the corresponding decoded image , so as to integrate and encapsulate the respective decoded images and the header messages of the respective decoded images into a single image data packet.

In an embodiment of the present invention, the image parameter further includes an image sequence number, and the encapsulation module further includes: inserting the header message into the header of the corresponding decoded image, and arranging the images with the header message according to the image sequence number. Decode the image.

In an embodiment of the present invention, the header message further includes a start character and an end character, and each of the image processing modules parses and processes the received image data packet includes: determining the received image data packet according to the start character and the end character. position of the header message, and the position of the decoded image is determined according to the length, height and number of channels.

In an embodiment of the present invention, the apparatus includes: using a publish/subscribe message mode to send the image data packet to a plurality of mutually independent image processing modules.

An electronic device includes a processor, a memory, and instructions stored in the memory, wherein the instructions, when executed by the processor, implement the method as described above.

A computer-readable storage medium having stored thereon computer instructions that, when executed, perform a method as described above.

Description of drawings

The following drawings are only intended to illustrate and explain the present invention schematically, and do not limit the scope of the present invention. in,

1 is a flowchart of an image transmission method according to an embodiment of the present invention;

2 is a schematic diagram of an image transmission method according to an embodiment of the present invention;

3 is a schematic diagram of an image transmission apparatus according to an embodiment of the present invention;

FIG. 4 is a schematic diagram of an electronic device according to an embodiment of the present invention.

Description of reference numerals

100 Image Transmission Methods

110-130 steps

210 Image sources

220 decoding module

230 package modules

240 MQTT Brokers

250 Image Processing Modules

251 The first image processing module

252 Second image processing module

253 The third image processing module

254 Fourth image processing module

300 Image Transmission Device

310 decoding module

320 package module

330 sending module

400 Electronic Devices

410 processors

420 memory

Detailed ways

In order to have a clearer understanding of the technical features, objects and effects of the present invention, the specific embodiments of the present invention will now be described with reference to the accompanying drawings.

Numerous specific details are set forth in the following description to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways than those described herein, and thus the present invention is not limited by the specific embodiments disclosed below.

As shown in this application and in the claims, unless the context clearly dictates otherwise, the words "a", "an", "an" and/or "the" are not intended to be specific in the singular and may include the plural. Generally speaking, the terms "comprising" and "comprising" only imply that the clearly identified steps and elements are included, and these steps and elements do not constitute an exclusive list, and the method or apparatus may also include other steps or elements.

FIG. 1 is a flowchart of an image transmission method 100 according to an embodiment of the present invention. As shown in FIG. 1, the image transmission method 100 includes:

Step 110: Decode the received image stream to obtain multiple decoded images and image parameters corresponding to each decoded image in the multiple decoded images.

The image stream can be captured by a web camera or an IP camera. The image stream includes multiple coded images (for example, JPEG or PNG format) with chronological order. The multiple coded images in the image stream are stored in the memory of the camera, or stored in the camera. in a file server in the network. Decode the received image stream to obtain the same number of decoded images, and the decoded images can be in RGB/RGBA/YUV format. It can be understood that the multiple encoded images in the image stream are of the same format, and the decoded images of multiple frames obtained by decoding are also of the same format. For example, the encoded images in the image stream are all in JPEG format, and the decoded images obtained by decoding are all in RGB format. After decoding the image stream, image parameters corresponding to the decoded image can also be obtained, and the image parameters include the length, height and number of channels of the decoded image. Taking a 4K RGB format image as an example, the length of an image is 32, the height is 20, and the number of channels is 3. In some embodiments, the image parameters may also include additional information such as camera ID, image serial number, and pipeline UUID.

FIG. 2 is a schematic diagram of an image transmission method according to an embodiment of the present invention. As shown in FIG. 2 , the image stream 210 is input to the decoding module 220, and the encoding module 220 decodes multiple encoded images in the image stream 210, and outputs the same number of decoded images as the encoded images and image parameters of each decoded image.

Step 120: Generate header messages of each decoded image according to the image parameters, and integrate and encapsulate each decoded image and the header message of each decoded image into a single image data packet.

The header message of each decoded image is generated according to the image parameters. The image parameters include the length, height and number of channels of the decoded image. Therefore, the location and data amount of the decoded image can be determined by the image parameters in the header message. In some embodiments, the header message may also include start and end characters to determine the location of the header message.

The following code 1 shows the code representation of the header message in an embodiment of the present invention. As shown in the code 1, the data structure of the header message FrameHeader includes the length (width), height (height), and number of channels of the decoded image. (channel), also includes camera id (source id), image serial number (frame_num), pipeline uuid (pipeline uuid), the start character is defined as VAS_, and the end character is defined as VAS$.

Code 1 The data structure code of the header message

After the header message of each decoded image is generated, each decoded image and the header message of each decoded image are integrated and encapsulated into a single image data packet. In some embodiments, integrating and encapsulating the respective decoded images and the header messages of the respective decoded images into a single image data packet includes: inserting the header message into the header of the corresponding decoded image, so as to encapsulate the respective decoded images and the header messages of the respective decoded images Consolidate encapsulation into a single image packet. Specifically, the header message of the first image is inserted into the header of the first image, the header message of the second image is inserted into the header of the second image, and so on, until the header message of the last image is inserted into the header of the last image It forms a single image data packet in which the header message and the image alternately appear, thus completing the integrated encapsulation of the decoded image and the header message.

The following code 2 shows a code schematic of an image data packet in an embodiment of the present invention. As shown in code 2, the data structure of the image data packet FramePayload includes a header file FrameHeader and a decoded image located after the header file FrameHeader.

typedef struct_FramePayload

{

FrameHeader header;

char data[IMAGE_MAX_WIDTH*IMAGE_MAX_HEIGHT*IMAGE_MAX_CHANNELS];

}FramePayload

Code 2 The data structure code of the image data packet

In some embodiments, the image parameter further includes an image sequence number, and the image transmission method 100 further includes: inserting a header message into the header of the corresponding decoded image, and arranging the decoded images with the header message according to the image sequence number. For example, a header message is inserted into the header of the corresponding decoded image, the image parameters in the header message include image sequence numbers, and the decoded images with header messages are arranged in ascending order according to the image sequence numbers from small sequence numbers to large sequence numbers.

In FIG. 2, a plurality of decoded images and corresponding image parameters output by the decoding module 220 are input to the encapsulation module 230. The encapsulation module 230 generates a header message of each decoded image according to the image parameters, and converts each decoded image and the header of each decoded image. The message is integrated and encapsulated into a single image data packet, and a single image data packet is output.

Step 130: Send the image data packets to a plurality of mutually independent image processing modules, and each image processing module parses and processes the received image data packets.

The multiple image processing modules are independent of each other, that is, each image processing module operates independently and does not interfere with each other. Multiple image processing modules may be located in multiple pieces of hardware in a distributed system. As shown in FIG. 2, the image processing module 250 may include a first image processing module 251, a second image processing module 252, a third image processing module 253, a fourth image processing module 254, etc. The first image processing module 251 may be used for Crop the image, the second image processing module can be used for feature extraction, the third image processing module can be used for image transformation, the fourth image processing module can be used for image modeling, etc. Through multiple independently running image processing modules, mutual interference between the image processing modules can be avoided, and each image processing module only needs to perform a single function, reducing the hardware requirements for each image processing module, thereby improving the image quality. Handle system scalability.

The image data packets are sent to a plurality of mutually independent image processing modules, and after receiving the image data packets, the image processing modules parse and process the received image data packets. Since the image data packet contains the header message, the image processing module can determine the data amount and position of the decoded image by reading the header message, which avoids the need for decoding by each image processing module, and reduces the computational complexity of the system. hardware requirements.

Specifically, the first image processing module 251 receives the image data packet, first reads the first header message (from VAS_ to VAS$), parses out the image parameters in the header message, and the length, height and number of channels are 32 respectively , 20 and 3, the data volume of the decoded image corresponding to the first header message is 32*20*3=1920, from which the effective data volume of the first image can be determined, and so on, until the last one in the image data packet Image, the position of each image in the image data package has been completed so far, the image processing module 251 can perform corresponding image processing functions on a single image, the second image processing module 252, the third image processing module 253, the fourth image processing module 254, etc. Similar operations to the image processing module 251 are also performed.

In some embodiments, the header message further includes a start character and an end character, and each image processing module parses and processes the received image data packet including: determining the position of the header message according to the start character and the end character, and determining the position of the header message according to the length, height and The number of channels determines the position of the decoded image.

In some embodiments, the image transmission method 100 may further include: using a publish/subscribe message mode to send the image data packets to a plurality of mutually independent image processing modules. Preferably, the MQTT protocol (Message Queuing Telemetry Transport, Message Queuing Telemetry Transport Protocol) can be used to send the image data packets to a plurality of mutually independent image processing modules. By using the publish/subscribe message mode to send image data packets to multiple independent image processing modules, it can ensure that the image data packets are sent to the image processing modules to prevent packet loss. In FIG. 2, the MQTT broker 240 sends the image data packets to the first image processing module 251, the second image processing module 252, the third image processing module 253, the fourth image processing module 254, etc., using the MQTT protocol, which can ensure the image data The packet is sent to the image processing module.

The embodiment of the present invention provides an image transmission method. By encapsulating the decoded image and the header message of the decoded image into image data packets and sending them to a plurality of mutually independent image processing modules, only one decoding is required for multiple mutually independent image processing modules. The image processing module is used for image processing, which reduces the computational complexity of the system and the requirements for hardware.

FIG. 3 is a schematic diagram of an image transmission apparatus 300 of an industrial control network according to an embodiment of the present invention. As shown in FIG. 3, the image transmission device 300 includes:

The decoding module 310 decodes the received image stream to obtain multiple decoded images and image parameters corresponding to each decoded image in the multiple decoded images. The image parameters include the length, height and number of channels of the decoded image, and the image stream includes multiple decoded images. encoded image.

The encapsulation module 320 generates a header message of each decoded image according to the image parameters, and integrates and encapsulates each decoded image and the header message of each decoded image into a single image data packet.

The sending module 330 sends the image data packets to a plurality of mutually independent image processing modules, and each image processing module parses and processes the received image data packets.

In some embodiments, the encapsulation module 320 integrates and encapsulates the respective decoded images and the header messages of the respective decoded images into a single image data packet, including: inserting the header message into the header of the corresponding decoded image, so as to encapsulate the respective decoded images and the respective decoded images The header message is integrated and encapsulated into a single image packet.

In some embodiments, the image parameter further includes an image sequence number, and the encapsulation module 320 further includes: inserting a header message into the header of the corresponding decoded image, and arranging the decoded images with the header message according to the image sequence number.

In some embodiments, the header message further includes a start character and an end character, and each image processing module parses and processes the received image data packet including: determining the position of the header message according to the start character and the end character, and determining the position of the header message according to the length, height and channel The number determines the position of the decoded image.

In some embodiments, the sending module 330 uses a publish/subscribe message mode to send the image data packets to a plurality of mutually independent image processing modules.

The present invention also provides an electronic device 400 . FIG. 4 is a schematic diagram of an electronic device 400 according to an embodiment of the present invention. As shown in FIG. 4 , the electronic device 400 includes a processor 410 and a memory 420. The memory 420 stores instructions stored therein, wherein the instructions, when executed by the processor 410, implement the method 100 as described above.

The present invention also proposes a computer-readable storage medium having computer instructions stored thereon, the computer instructions, when executed, perform the method 100 as described above.

Some aspects of the methods and apparatus of the present invention may be performed entirely by hardware, entirely by software (including firmware, resident software, microcode, etc.), or by a combination of hardware and software. The above hardware or software may be referred to as a "data block", "module", "engine", "unit", "component" or "system". The processor may be one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DAPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), processors , controller, microcontroller, microprocessor, or a combination thereof. Furthermore, aspects of the present invention may be embodied as a computer product comprising computer readable program code embodied in one or more computer readable media. For example, computer readable media may include, but are not limited to, magnetic storage devices (eg, hard disks, floppy disks, magnetic tapes...), optical disks (eg, compact discs (CDs), digital versatile discs (DVDs)...), smart cards and flash memory devices (eg cards, sticks, key drives...).

Flowcharts are used herein to illustrate operations performed by methods according to embodiments of the present application. It should be understood that the preceding operations are not necessarily performed in exact order. Rather, the various steps may be processed in reverse order or concurrently. At the same time, other actions are either added to these processes, or a step or steps are removed from these processes.

It should be understood that although this specification is described according to various embodiments, not each embodiment only includes an independent technical solution, and this description in the specification is only for the sake of clarity, and those skilled in the art should take the specification as a whole , the technical solutions in each embodiment can also be appropriately combined to form other implementations that can be understood by those skilled in the art.

The above descriptions are only exemplary embodiments of the present invention, and are not intended to limit the scope of the present invention. Any equivalent changes, modifications and combinations made by any person skilled in the art without departing from the concept and principles of the present invention shall fall within the protection scope of the present invention.

Claims

An image transmission method (100), characterized in that the image transmission method (100) comprises:

Decode the received image stream to obtain multiple decoded images and image parameters corresponding to each decoded image in the multiple decoded images, where the image parameters include the length, height and number of channels of the decoded image, the image the stream includes a plurality of encoded pictures (110);

Generate header messages of the respective decoded images according to the image parameters, and integrate and encapsulate the respective decoded images and the header messages of the respective decoded images into a single image data packet (120);

The image data packets are sent to a plurality of mutually independent image processing modules, and each of the image processing modules parses and processes the received image data packets (130).
The method (100) according to claim 1, wherein integrating and encapsulating the respective decoded images and the header messages of the respective decoded images into a single image data packet comprises: inserting the header messages into the corresponding decoded images , so as to integrate and encapsulate the respective decoded images and the header messages of the respective decoded images into a single image data packet.
The method (100) according to claim 2, wherein the image parameter further comprises an image sequence number, and the method (100) further comprises: inserting the header message into the header of the corresponding decoded image, according to the The picture sequence number aligns the decoded picture with the header message.
The method (100) according to claim 1, wherein the header message further includes a start character and an end character, and each of the image processing modules parses and processes the received image data packet comprising: according to the The start character and end character determine the position of the header message, and the position of the decoded image according to the length, height and number of channels.
The method (100) according to claim 1, characterized in that, the method (100) comprises: using a publish/subscribe message mode to send the image data packet to a plurality of mutually independent image processing modules.
An image transmission device (300), characterized in that the image transmission device (300) comprises:

The decoding module (310) decodes the received image stream to obtain multiple decoded images and image parameters corresponding to each decoded image in the multiple decoded images, the image parameters including the length, height and the number of channels, the image stream includes a plurality of encoded images;

an encapsulation module (320), generating header messages of the respective decoded images according to the image parameters, and integrating and encapsulating the respective decoded images and the header messages of the respective decoded images into a single image data packet;

A sending module (330) sends the image data packets to a plurality of mutually independent image processing modules, and each of the image processing modules parses and processes the received image data packets.
The apparatus (300) according to claim 6, wherein the encapsulation module (320) integrates and encapsulates the respective decoded images and the header messages of the respective decoded images into a single image data packet, comprising: encapsulating the The header message is inserted into the header of the corresponding decoded image, so as to integrate and encapsulate the respective decoded images and the header messages of the respective decoded images into a single image data packet.
The apparatus (300) according to claim 7, wherein the image parameter further comprises an image sequence number, and the encapsulation module (320) further comprises: inserting the header message into the header of the corresponding decoded image, according to The picture sequence numbers rank decoded pictures with header messages.
The apparatus (300) according to claim 6, wherein the header message further includes a start character and an end character, and each of the image processing modules parses and processes the received image data packets comprising: according to the The start character and end character determine the position of the header message, and the position of the decoded image according to the length, height and number of channels.
The apparatus (300) according to claim 6, characterized in that, the apparatus (300) comprises: using a publish/subscribe message mode to send the image data packet to a plurality of mutually independent image processing modules.
An electronic device (400) comprising a processor (410), a memory (420) and instructions stored in the memory (420), wherein the instructions when executed by the processor (410) implement the claims The method of any one of 1-5.
A computer-readable storage medium having stored thereon computer instructions that, when executed, perform the method of any of claims 1-5.