CN111193901B - Image transmission method, imaging device, system and vehicle - Google Patents
Image transmission method, imaging device, system and vehicle Download PDFInfo
- Publication number
- CN111193901B CN111193901B CN201811362126.8A CN201811362126A CN111193901B CN 111193901 B CN111193901 B CN 111193901B CN 201811362126 A CN201811362126 A CN 201811362126A CN 111193901 B CN111193901 B CN 111193901B
- Authority
- CN
- China
- Prior art keywords
- data stream
- image data
- image
- pixel
- sub
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/10—Cameras or camera modules comprising electronic image sensors; Control thereof for generating image signals from different wavelengths
Abstract
The invention discloses an image transmission method, imaging equipment, an imaging system and a vehicle, wherein the method comprises the following steps: obtaining an image data stream; separating the data of each pixel of each frame of image in the image data stream to form a main image data stream and a secondary image data stream; the main image data stream comprises a main image of each frame of image, is used for expressing the main content of the image and can be directly used for displaying; writing the separated main image data stream and sub image data stream into a memory; a primary image data stream is read from a memory and transmitted to a receiving device for image display by the receiving device in accordance with the primary image data stream. The invention can reduce the operation burden of the CPU of the receiving equipment and realize faster image display.
Description
Technical Field
The present invention relates to the field of image processing technologies, and in particular, to an image transmission method, an imaging device, an imaging system, and a vehicle.
Background
With the development of the intelligent industry, robot obstacle avoidance, Simultaneous Localization And Mapping (SLAM) technology And unmanned technology, imaging devices (such as cameras) are widely used in these fields. Most of the imaging devices currently employ Charge Coupled Devices (CCD) and Complementary Metal-Oxide-Semiconductor (CMOS) image sensors, and CMOS image sensors occupy more and more important positions in the field of image processing by virtue of their characteristics of low cost, low power consumption, high integration degree, and the like.
Currently, most CMOS image sensors on the market output Bayer (Bayer) images, and each color component in each pixel of most Bayer images occupies 12 bits (bit) depth, has 4096 color levels, and has more information than 8bit (having 256 color levels) images. Therefore, in order to obtain more information, the imaging device is required to transmit all the 12-bit data of each color component of each pixel to a receiving device (such as a computer, a server, etc.), so as to perform operations of extracting, identifying, etc. the image content. However, in general, an image needs to be displayed on a display device in time for a person to watch, and then whether the imaging device is focused correctly, whether a window is accurate, and the like are judged from a subjective angle. However, most display devices (general-purpose displays) currently support display of images of only 8bit color component depth, and images of 12bit color component depth are difficult to directly display on a display device. Therefore, there is a need for a transmission method of image data so that a display device of a receiving device can conveniently perform image display.
Disclosure of Invention
In view of the technical drawbacks and disadvantages of the prior art, embodiments of the present invention provide a method, an imaging apparatus, a system, and a vehicle for transmitting an image, which overcome or at least partially solve the above problems.
As an aspect of the embodiments of the present invention, there is provided an image transmission method including:
obtaining an image data stream;
separating the data of each pixel of each frame of image in the image data stream to form a main image data stream and a secondary image data stream; the main image data stream comprises a main image of each frame of image, is used for expressing the main content of the image and can be directly used for displaying;
writing the separated main image data stream and sub image data stream into a memory;
a primary image data stream is read from a memory and transmitted to a receiving device for image display by the receiving device in accordance with the primary image data stream.
In one embodiment, the method further comprises:
receiving an auxiliary graph demand instruction sent by receiving equipment;
and reading the secondary image data stream from the memory and sending the secondary image data stream to the receiving device, so that the receiving device performs bit operation on the data of each pixel of each frame of image in the main image data stream and the secondary image data stream to obtain the image data stream.
In one embodiment, the separating the data of the pixels of each frame of image in the image data stream to form the main image data stream and the sub-image data stream includes:
separating each color component data of each pixel in each frame image in the image data stream according to high M bits and low N bits; wherein M + N is greater than a predicted maximum allowed transmission bit number threshold;
forming a main image data stream based on the high M-bit data separated from the color component data of each pixel; wherein M is less than or equal to a predicted maximum allowed transmission bit number threshold;
a sub-picture image data stream is formed based on the lower N-bit data separated from the respective color component data in each pixel.
In one embodiment, the forming of the main image data stream based on the high M-bit data separated from the respective color component data of each pixel includes:
forming a main image data stream in the order of pixels by using the high M-bit data separated from each color component data of each pixel as the data of the corresponding pixel in the main image data stream;
the forming of a sub-picture image data stream based on the low N-bit data separated from the respective color component data in each pixel includes:
taking the low N-bit data separated from each color component data of each pixel as the data of the corresponding pixel in the sub-image data stream, and forming the sub-image data stream according to the sequence of the pixels; wherein N is less than or equal to a predicted maximum allowed transmission bit number threshold;
or, each will
The low N-bit data separated from each color component data in each adjacent pixel is combined as data of a corresponding pixel in the sub-image data stream, and the sub-image data stream is formed in the order of pixels.
In one embodiment, the relationship between M and N is: m ═ α N, α is a predetermined multiple, α is 1 or more, and M and N are natural numbers.
In one embodiment, M-8, N-8; or M is 8 and N is 4.
As another aspect of an embodiment of the present invention, there is provided an image forming apparatus including: an image sensor, a processor, and a memory;
the image sensor is used for collecting images and forming an image data stream;
the processor to obtain an image data stream; separating the data of each pixel of each frame of image in the image data stream to form a main image data stream and a secondary image data stream; the main image data stream comprises a main image of each frame of image, is used for expressing the main content of the image and can be directly used for displaying; writing the separated main image data stream and sub image data stream into a memory; a primary image data stream is read from a memory and transmitted to a receiving device for image display by the receiving device in accordance with the primary image data stream.
In one embodiment, the processor is further configured to:
receiving an auxiliary graph demand instruction sent by receiving equipment;
and reading the secondary image data stream from the memory and sending the secondary image data stream to the receiving device, so that the receiving device performs bit operation on the data of each pixel of each frame of image in the main image data stream and the secondary image data stream to obtain the image data stream.
In one embodiment, the processor is specifically configured to:
separating each color component data of each pixel in each frame image in the image data stream according to high M bits and low N bits; wherein M + N is greater than a predicted maximum allowed transmission bit number threshold;
forming a main image data stream based on the high M-bit data separated from the color component data of each pixel; wherein M is less than or equal to a predicted maximum allowed transmission bit number threshold;
a sub-picture image data stream is formed based on the lower N-bit data separated from the respective color component data in each pixel.
In one embodiment, the processor is specifically configured to:
forming a main image data stream in the order of pixels by using the high M-bit data separated from each color component data of each pixel as the data of the corresponding pixel in the main image data stream;
the processor is specifically configured to:
taking the low N-bit data separated from each color component data of each pixel as the data of the corresponding pixel in the sub-image data stream, and forming the sub-image data stream according to the sequence of the pixels; wherein N is less than or equal to a predicted maximum allowed transmission bit number threshold;
or, each will
The low N-bit data separated from each color component data in each adjacent pixel is combined as data of a corresponding pixel in the sub-image data stream, and the sub-image data stream is formed in the order of pixels.
In one embodiment, the processor is a field programmable gate array.
In one embodiment, the memory stores the primary image data stream and the secondary image data stream separately.
As a further aspect of an embodiment of the present invention, there is provided an image transmission system including the above-described imaging device and receiving device;
and the receiving device is used for receiving the main image data stream sent by the imaging device and displaying images according to the main image data stream.
In one embodiment, the receiving device is further configured to:
sending a sub-map requirement instruction to the imaging equipment;
and receiving a secondary image data stream sent by an imaging device, and performing bit operation on the data of each pixel of each frame of image in the main image data stream and the secondary image data stream to obtain an image data stream.
As a final aspect of the embodiment of the present invention, there is provided a vehicle including the above-described image forming apparatus and receiving apparatus;
and the receiving device is used for receiving the main image data stream sent by the imaging device and displaying images according to the main image data stream.
In one embodiment, the receiving device is further configured to:
sending a sub-map requirement instruction to the imaging equipment;
and receiving a secondary image data stream sent by an imaging device, and performing bit operation on the data of each pixel of each frame of image in the main image data stream and the secondary image data stream to obtain an image data stream.
The embodiment of the invention at least realizes the following technical effects:
according to the image transmission method, the imaging device, the system and the vehicle provided by the embodiment of the invention, the separation of the data of each pixel of each frame of image in the image data stream can be realized in a processor chip (such as an FPGA) with an image processing capacity, a main image data stream and a secondary image data stream are formed, and the transmission of the image data stream is realized after the main image data stream and the secondary image data stream are cached in a memory. According to the embodiment of the invention, according to the requirement of the receiving equipment, when the receiving equipment only needs to receive and display the main image, only the data of the data stream of the main image needs to be transmitted to the receiving equipment, so that for some receiving equipment with low CPU performance, the transmitted data volume is reduced, the operation burden of the CPU can be reduced undoubtedly, and faster image display is realized.
Additional features and advantages of the invention will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. The objectives and other advantages of the invention will be realized and attained by the structure particularly pointed out in the written description and claims hereof as well as the appended drawings.
The technical solution of the present invention is further described in detail by the accompanying drawings and embodiments.
Drawings
The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this specification, illustrate embodiments of the invention and together with the description serve to explain the principles of the invention and not to limit the invention. In the drawings:
FIG. 1 is a schematic diagram of a prior art image pixel data storage;
FIG. 2 is a flowchart of an image transmission method according to an embodiment of the present invention;
FIG. 3 is a flowchart of an image transmission method for transmitting a complete image data stream according to an embodiment of the present invention;
FIG. 4 is a flow chart of image pixel data separation according to an embodiment of the present invention;
FIG. 5 is a first schematic diagram illustrating storage of a primary image data stream and a secondary image data stream according to an embodiment of the present invention;
FIG. 6 is a second schematic diagram of the storage of the main image data stream and the sub image data stream according to the embodiment of the present invention;
fig. 7 is a structural view of an image forming apparatus provided in an embodiment of the present invention;
fig. 8 is a schematic diagram of an image transmission system according to an embodiment of the present invention.
Detailed Description
Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.
In order to make the present invention better understood by those skilled in the art, some technical terms related to the present invention are explained below:
FPGA: Field-Programmable Gate Array.
Bayer image: that is, a bayer image is an image formed using a bayer array, which is one of the main technologies for realizing a CCD or CMOS sensor to capture a color image.
A CCD image sensor: charge Coupled Device, ccd image sensor.
A CMOS image sensor: complementary Metal-Oxide-Semiconductor (CMOS) image sensors.
DMA: direct Memory Access, can be implemented by hardware devices of different speeds.
DDR: double Data Rate, Double Data synchronous dynamic random access memory, is one of the memories.
In implementing the embodiments of the present invention, the inventors found that the transmission of image data can be performed in the following manner in general:
as shown in fig. 1, taking transmission of Bayer image data with a depth of 12 bits as an example, due to transmission requirements of network data, 1 byte can generally store 8 bits of data, and in general, for data transmission, only storage of image data is concerned, that is, each 12 bits of pixel data is split and spliced, so that each byte stores 8 bits of data. For 12-bit image data, it is generally necessary to split and join data of each pixel to form whole byte data, and to mashup the data together before transmission. For example P in FIG. 11_8bit、P2_8bit、P3_8bit、P4_8bitRespectively 1 st to 4 th pixels (only 4 pixels are taken as an example here, it should be noted that, in a frame of image data, several pixel data may be included), 8bit of data, and P1_4bit、P2_4bit、P3_4bit、P4_4bitData of another 4 bits of 1 st to 4 th pixels, respectively, wherein P1_8bit、P2_8bit、P3_8bit、P4_8bitOccupy 1 byte to store, P, respectively1_4bitAnd P2_4bitSpliced together to take 1 byte to store, P3_4bitAnd P4_4bitThe splicing together occupies 1 byte for storage, and by analogy, the pixel data in the whole frame of image data can be split and spliced to form a structure similar to that shown in fig. 1 for storage.
Thus, when the receiving device needs to display an image, the receiving device needs to traverse each pixel from the above structure, recover the data of each pixel back to 12-bit image data, and convert the data into 8-bit image data in a linear stretching or nonlinear mapping manner for display. This increases the burden on a Central Processing Unit (CPU) of the receiving apparatus to some extent, and for a receiving apparatus with lower performance, there is a risk of affecting image display. In addition, the mode of splicing and mashup transmission of the pixel data in the image also increases the difficulty of traversing and recovering each pixel, so that the processing of the receiving equipment is slow, and the real-time display of the image is difficult.
Various specific embodiments of the image transmission method, the imaging device, the system and the vehicle according to the embodiments of the present invention are described in detail below.
Referring to fig. 2, a method for transmitting an image according to an embodiment of the present invention includes the following steps:
and S21, obtaining an image data stream.
And S22, separating the data of each pixel of each frame image in the image data stream to form a main image data stream and a secondary image data stream.
The main image data stream comprises a main image of each frame of image, and the main image data stream is used for expressing the main content of the image and can be directly used for displaying.
And S23, writing the separated main image data stream and the sub image data stream into a memory.
S24, reading the main image data stream from the memory and transmitting to the receiving device to cause the receiving device to display an image in accordance with the main image data stream.
According to the image transmission method provided by the embodiment of the invention, when the receiving device only needs to display the image in a certain time period and no requirement or low requirement is provided for the display quality of the image, only the main image data stream needs to be sent to the receiving device for the operation processes of display, processing and the like of the receiving device, so that the transmitted data volume is reduced, the operation load of a CPU (central processing unit) of the receiving device can be reduced, and faster image display is realized.
The main image according to the embodiment of the present invention generally includes most of the information of the image, and is used to express the main content of the image, and can be directly used for display. The secondary image, in general, contains some detail information of the image that complements the content of the primary image, and cannot be used directly for display.
In the embodiment of the present invention, each frame of image in the image data stream is formed by combining a plurality of pixels, each pixel further includes a plurality of color components, and data of each color component occupies a certain byte. The obtained image data may be various, for example, an image with a depth of 8bit color component, an image with a depth of 12bit color component, an image with a depth of 16bit color component, and the like, and the embodiment of the present invention is not limited thereto.
For convenience of description, an image of the Tbit color component depth (the size of T is equal to the depth of each color component in a pixel, and the size may be 4, 8, 12 or other numerical values), which is simply referred to as Tbit, for example, an image of the 12-bit color component depth is simply referred to as 12-bit, although the specific size of the color component depth is not limited in the embodiments of the present invention.
In one embodiment, when the receiving device needs to display the entire content of the image or apply the original image data stream, the obtained image data stream needs to be completely transmitted to the receiving device. Specifically, referring to fig. 3, the method includes the following steps:
and S31, receiving the sub-map requirement instruction sent by the receiving equipment.
And S32, reading the sub-image data stream from the memory and sending the sub-image data stream to the receiving device, so that the receiving device performs bit operation on the data of each pixel of each frame of image in the main image data stream and the sub-image data stream to obtain an image data stream.
Here, the receiving device, upon receiving the data of the respective pixels of each frame image in the main image data stream and the sub image data stream, performs bit arithmetic processing on the received data so that the main image data stream and the sub image data stream in the separated state can be restored to the original image data streams.
Thus, the receiving device can recover the original image data stream through simple bit operation, and the subsequent processing (such as storage, display, data analysis, etc.) of the receiving device is facilitated. In addition, the speed of bit operation is the fastest for most of the CPUs of the receiving devices, so that the operation load of the CPUs of the receiving devices can be reduced. For a lower performance receiving device, the risk of image display can be reduced.
Compared with the method of splicing and mashup transmission of the pixel data in the image, the method separates the data of each pixel of each frame of image in the image data stream, and selects whether to read out the sub-image data stream from the memory according to the requirement of the receiving equipment. The method avoids the data of all pixels in the image from being spliced and mashup together for transmission, reduces the difficulty of traversing and recovering each pixel, and enables the receiving equipment to process more quickly, thereby displaying the image in real time.
In one embodiment, referring to fig. 4, in the step S22, the data of each pixel of each frame image in the image data stream is separated to form the main image data stream and the sub-image data stream, which can be implemented as follows:
s41, separating each color component data of each pixel in each frame image in the image data stream according to high M bits and low N bits; wherein M + N is greater than a predicted maximum allowed transmission bit number threshold.
S42, a main image data stream is formed based on the high M-bit data separated from the color component data of each pixel. Wherein M is less than or equal to a predicted maximum allowed transmission bit number threshold.
S43, a sub-picture data stream is formed based on the low N-bit data separated from each color component data in each pixel.
It should be noted that, the manner of separating the data of each pixel of each frame image in the image data stream may be various, for example, the data may be separated according to the upper M bits and the lower N bits of each color component, or may be separated according to the lower M bits and the upper N bits of each color component, or may be a separation manner in which the N bits in each color component are a main image data stream, and the other bits are a sub image data stream, and the like, and the embodiment of the present invention is not limited thereto.
Here, the maximum allowable transmission bit number threshold refers to a maximum number of bits that data of each pixel of each frame of image in the image data stream can be allowed to be transmitted during the image transmission process; or the maximum number of bits that the receiving device can allow for the display of data for individual pixels of each frame of image in the image data stream. For example, in network data transmission, each Byte (Byte) is generally transmitted according to 8 bits, and the maximum allowed transmission bit number threshold may be 8.
In the embodiment of the present invention, the value of M + N is generally greater than the predicted maximum allowable transmission bit number threshold; if the value of M + N is less than or equal to the predicted maximum allowable transmission bit number threshold value, the image data stream does not need to be separated and can be directly used for image transmission and display. In addition, both M and N should be equal to or less than the predetermined maximum allowed transmission bit number threshold, i.e., the maximum value of M and N should not exceed the predetermined maximum allowed transmission bit number threshold.
In one embodiment, the formation of the main image data stream based on the high M-bit data separated from the color component data of each pixel in step S42 described above may be implemented by:
the high M-bit data separated from the color component data of each pixel is used as the data of the corresponding pixel in the main image data stream, and the main image data stream is formed according to the order of the pixels.
The sub-image data stream formed in step S43 based on the low-N bit data separated from the color component data in each pixel may be implemented in one of the following two ways, see the following first way and second way:
the first method is as follows: taking the low N-bit data separated from each color component data of each pixel as the data of the corresponding pixel in the sub-image data stream, and forming the sub-image data stream according to the sequence of the pixels; wherein N is less than or equal to a predicted maximum allowed transmission bit number threshold.
Or adopting a second mode: will each be
The low N-bit data separated from each color component data in adjacent pixels are combined to form sub-image data stream, and the sub-image data stream is formed in the order of pixelsLike the data stream.
In the embodiment of the present invention, each color component data of each pixel in each frame image in the image data stream is described in detail in a manner of separating the high M bits and the low N bits.
In one embodiment, assume that the relationship of M and N is: m ═ α N, α is a predetermined multiple, α is 1 or more, and M and N are natural numbers.
In one embodiment, M-8, N-8; or M is 8 and N is 4.
Of course, the embodiments of the present invention do not limit the specific values of α, M, and N, and as long as all the values of the above technical solutions provided by the embodiments of the present invention are feasible, other possible cases of α, M, and N are not listed here.
For ease of understanding, two practical examples are illustrated:
example one:
when α is 1, M is 8, and N is 8, the obtained image data stream is a 16-bit image. Separating each color component data of each pixel in each frame image in the 16-bit image data stream according to the high 8 bits and the low 8 bits; forming a main image data stream in the order of pixels by using the high 8-bit data separated from each color component data of each pixel as the data of the corresponding pixel in the main image data stream; the lower 8-bit data separated from each color component data of each pixel is used as data of a corresponding pixel in the sub-picture image data stream, and the sub-picture image data stream is formed in the order of pixels.
The obtained 16-bit image data stream is separated into a main image data stream of 8 bits high and a sub-image data stream of 8 bits low. According to the method of carrying out data transmission and storage by taking bytes as units, the main image data stream with 8 bits high or the secondary image data stream with 8 bits low can be directly used for data transmission and storage.
Referring to fig. 5, for an image with an original size of 16bit in an image data stream, the separated main image data stream and sub-image data stream are stored in the memory in the following pixel order:
a1 is the high 8-bit data for the first pixel, and A2 is the high 8-bit data … … An for the second pixel, which is the high 8-bit data for the nth pixel.
B1 is the low 8-bit data of the first pixel, and B2 is the low 8-bit data … … Bn of the second pixel is the low 8-bit data of the nth pixel.
Wherein the pixel value of each pixel in the main image data stream is:
C1＝A1；
C2＝A2；
……
Cn＝An；
in the above formula, C1 is the pixel value of the first pixel in the main image, and C2 is the pixel value … … Cn of the second pixel in the main image is the pixel value of the nth pixel in the main image.
The pixel values of the pixels in the sub-image data stream are:
C1′＝B1；
C2′＝B2；
……
Cn′＝Bn；
in the above equation, C1 ' is the pixel value of the first pixel in the sub-map image, and C2 ' is the pixel value … … Cn ' of the second pixel in the sub-map image is the pixel value of the nth pixel in the sub-map image.
Example two:
when α is 2, M is 8, and N is 4, the obtained image data stream is a 12-bit image. Separating each color component data of each pixel in each frame image in the 12-bit image data stream according to high M bits and low N bits; forming a main image data stream in the order of pixels by using the high 8-bit data separated from each color component data of each pixel as the data of the corresponding pixel in the main image data stream; and combining the low 4-bit data separated from each color component data in every two adjacent pixels to form the data of the corresponding pixel in the sub-image data stream, and forming the sub-image data stream according to the order of the pixels.
In this case, the obtained 12-bit image data stream is separated into a main image data stream of 8 bits higher and a sub-image data stream of 4 bits lower. The main image data stream with a high 8bit can be directly transmitted and stored, and the secondary image data stream with a low 4bit can be transmitted and stored after being combined into the image data stream with an 8 bit. That is, one pixel data is stored in one byte in the main image data stream; in the sub-image data stream, two adjacent pixel data of the original image are stored within one byte, and the size of the main image data stream is twice as large as that of the sub-image data stream. Thus, the transmission and storage of the combined sub-image data stream are also facilitated.
After separating all the obtained image data streams and forming a main image data stream and a sub image data stream, the FPGA writes data of the main image data stream and data of the sub image data stream in the memory in order of pixels. The memory may be DDR or any other memory device having the same function, and is not limited herein.
Referring to fig. 6, for an original 12bit image, the separated main image data stream and sub-image data stream are stored in the memory in pixel order as follows:
p1 is the high 8-bit data of the first pixel, P2 is the high 8-bit data … … Pn of the second pixel is the high 8-bit data of the nth pixel;
q1 is the low 4-bit data of the first pixel, Q2 is the low 4-bit data … … Qn of the second pixel is the low 4-bit data of the nth pixel;
m1 is the first set of 8bit data consisting of Q1 and Q2, M2 is the second set of 8bit data consisting of Q3 and Q4, … … Mn/2Is a combination of Qn-1 and Qn
Group 8bit data.
If the receiving device only needs to apply the main image, the data stream of the main image is transmitted to the receiving device. For example, the pixel values of the pixels of the main image stream are:
I1＝P1；
I2＝P2；
……
In＝Pn；
in the above formula, I1 is the pixel value of the first pixel In the main image, and I2 is the pixel value … … In of the second pixel In the main image is the pixel value of the nth pixel In the main image.
If the receiving device needs to apply the original image, the main image data stream and the sub-image data stream are both sent to the receiving device, and the receiving device performs bit operation on the data of the main image data stream and the sub-image data stream to obtain the pixel value of the original image data stream.
The process of the receiving device for bit operation is as follows:
I1′＝(P1<<4)|Q1；
I2′＝(P2<<4)|Q2；
……
In′＝(Pn<<4)|Qn；
in the above formula, I1 ' is the pixel value of the first pixel In the original image data stream obtained by bit operation, I2 ' is the pixel value … … In ' of the second pixel In the original image data stream obtained by bit operation is the pixel value of the nth pixel In the original image data stream obtained by bit operation; "Pn < < 4" indicates that the upper 8bit data of the nth pixel is shifted to the left by 4 bits to complement the 4bit data of Qn after Pn.
According to the image transmission method provided by the embodiment of the invention, the original image data stream is separated into the main image data stream and the sub-image data stream, and the main image data stream and the sub-image data stream are selectively sent to the receiving device according to the requirement of the receiving device, or the main image data stream and the sub-image data stream are sent to the receiving device, so that the situation that the data of each pixel of each frame image in the original image data stream are mixed together and sent to the receiving device is avoided, and according to the capability and the requirement of the receiving device, the transmission efficiency can be improved while the display quality of the image of the receiving device is ensured.
Referring to fig. 7, an image forming apparatus according to an embodiment of the present invention includes: an image sensor 71, a processor 72, and a memory 73.
An image sensor 71 for image acquisition, forming an image data stream;
a processor 72 for obtaining a stream of image data; separating the data of each pixel of each frame of image in the image data stream to form a main image data stream and a secondary image data stream; the main image data stream comprises a main image of each frame of image, is used for expressing the main content of the image and can be directly used for displaying; writing the separated main image data stream and sub image data stream into the memory 73; the main image data stream is read from the memory 73 and transmitted to the receiving device to enable the receiving device to display an image in accordance with the main image data stream.
In one embodiment, the processor 72 may be any of a variety of processor chips with image Processing capabilities, such as a conventional FPGA, a Digital Signal Processor (DSP), or other similar processor.
In one embodiment, processor 72 is further configured to: receiving an auxiliary graph demand instruction sent by receiving equipment; the sub-image data stream is read from the memory 73 and sent to the receiving device so that the receiving device performs a bit operation on the data of each pixel of each frame of image in the main image data stream and the sub-image data stream to obtain an image data stream.
In one embodiment, processor 72 is specifically configured to: separating each color component data of each pixel in each frame image in the image data stream according to high M bits and low N bits; wherein M + N is greater than a predicted maximum allowed transmission bit number threshold; forming a main image data stream based on the high M-bit data separated from the color component data of each pixel; wherein M is less than or equal to a predicted maximum allowed transmission bit number threshold; a sub-picture image data stream is formed based on the lower N-bit data separated from the respective color component data in each pixel.
In one embodiment, processor 72 is specifically configured to: the high M-bit data separated from each color component data of each pixel is used as the data of the corresponding pixel in the main image data stream according to the imageThe order of the pixels forms a main image data stream; taking the low N-bit data separated from each color component data of each pixel as the data of the corresponding pixel in the sub-image data stream, and forming the sub-image data stream according to the sequence of the pixels; wherein N is less than or equal to a predicted maximum allowed transmission bit number threshold; or, each will
The low N-bit data separated from each color component data in each adjacent pixel is combined as data of a corresponding pixel in the sub-image data stream, and the sub-image data stream is formed in the order of pixels.
In one embodiment, the memory 73 stores the primary image data stream and the secondary image data stream separately. Thereby facilitating the separate reading of the primary image data stream and the secondary image data stream while avoiding blending the two together.
An image transmission system according to an embodiment of the present invention, shown in fig. 8, includes an imaging device 81 and a receiving device 82. For a specific operation process of the imaging device 81, reference is made to the specific implementation of the imaging device corresponding to fig. 7, which is not described herein again. The receiving device 82 is configured to receive the main image data stream transmitted from the imaging device 81 and perform image display based on the main image data stream.
In one embodiment, the receiving device 82 is further configured to: transmitting a sub-map requirement instruction to the imaging device 81; and receiving the secondary image data stream sent by the imaging device 81, and performing bit operation on the data of each pixel of each frame of image in the main image data stream and the secondary image data stream to obtain an image data stream.
The receiving device may be, for example, a computer, a server, or other devices, and the embodiment of the present invention is not limited thereto.
In addition, the embodiment of the invention also provides a vehicle, which comprises the imaging device and the receiving device; for specific working processes of the imaging device and the receiving device, reference is made to the specific implementation of the imaging device and the receiving device corresponding to fig. 7 and fig. 8, which is not described herein again.
As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, optical storage, and the like) having computer-usable program code embodied therein.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
It will be apparent to those skilled in the art that various changes and modifications may be made in the present invention without departing from the spirit and scope of the invention. Thus, if such modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include such modifications and variations.
Claims (12)
1. A method for transmitting an image, comprising:
obtaining an image data stream;
separating the data of each pixel of each frame of image in the image data stream to form a main image data stream and a secondary image data stream; the main image data stream comprises a main image of each frame of image, is used for expressing the main content of the image and can be directly used for displaying;
writing the separated main image data stream and sub image data stream into a memory;
reading out a main image data stream from a memory and transmitting to a receiving device to cause the receiving device to perform image display based on the main image data stream,
wherein the separating the data of each pixel of each frame of image in the image data stream to form a main image data stream and a sub-image data stream comprises:
separating high M bit data from each color component data of each pixel to serve as data of a corresponding pixel in the main image data stream, and forming the main image data stream according to the sequence of the pixels, wherein M is less than or equal to a predicted maximum allowable transmission bit threshold value;
separating the color component data of each pixel into lower N bits of data to form a sub-image data stream, wherein M + N is greater than a predetermined maximum allowable transmission bit threshold,
the method for separating the color component data of each pixel into low N-bit data to form a sub-image data stream includes:
taking the low N-bit data separated from each color component data of each pixel as the data of the corresponding pixel in the sub-image data stream, and forming the sub-image data stream according to the sequence of the pixels; wherein N is less than or equal to a predicted maximum allowed transmission bit number threshold;
or, each will
The low N-bit data separated from each color component data in each adjacent pixel is combined as data of a corresponding pixel in the sub-image data stream, and the sub-image data stream is formed in the order of pixels.
2. The method of claim 1, further comprising:
receiving an auxiliary graph demand instruction sent by receiving equipment;
and reading the secondary image data stream from the memory and sending the secondary image data stream to the receiving device, so that the receiving device performs bit operation on the data of each pixel of each frame of image in the main image data stream and the secondary image data stream to obtain the image data stream.
3. The method of claim 1, wherein the relationship of M and N is: m ═ α N, α is a predetermined multiple, α is 1 or more, and M and N are natural numbers.
4. The method of claim 3, wherein M-8, N-8; or M is 8 and N is 4.
5. An image forming apparatus, characterized by comprising: an image sensor, a processor, and a memory;
the image sensor is used for collecting images and forming an image data stream;
the processor to obtain an image data stream; separating the data of each pixel of each frame of image in the image data stream to form a main image data stream and a secondary image data stream; the main image data stream comprises a main image of each frame of image, is used for expressing the main content of the image and can be directly used for displaying; writing the separated main image data stream and sub image data stream into a memory; reading out a main image data stream from a memory and transmitting to a receiving device to cause the receiving device to perform image display based on the main image data stream,
the processor is specifically configured to:
separating the respective color component data of each pixel into high M-bit data as data for a corresponding pixel in the main image data stream, forming the main image data stream in pixel order, wherein M is equal to or less than a predetermined maximum allowable transmission bit number threshold value,
the processor is specifically further configured to:
separating low N-bit data from each color component data of each pixel to serve as data of a corresponding pixel in the sub-image data stream, and forming the sub-image data stream according to the sequence of the pixels; wherein N is less than or equal to a predicted maximum allowed transmission bit number threshold;
or, each will
Separating each color component data in adjacent pixels into a lower N-bit data combination as data of corresponding pixels in the sub-image data stream, forming the sub-image data stream in the order of pixels,
wherein M + N is greater than a predicted maximum allowed transmission bit number threshold.
6. The imaging device of claim 5, wherein the processor is further to:
receiving an auxiliary graph demand instruction sent by receiving equipment;
and reading the secondary image data stream from the memory and sending the secondary image data stream to the receiving device, so that the receiving device performs bit operation on the data of each pixel of each frame of image in the main image data stream and the secondary image data stream to obtain the image data stream.
7. The imaging device of claim 5 or 6, wherein the processor is a field programmable gate array.
8. The imaging device of claim 5 or 6, wherein the memory stores the primary image data stream and secondary image data stream separately.
9. A transmission system of an image, comprising the imaging device and the receiving device according to any one of claims 5 to 8;
and the receiving device is used for receiving the main image data stream sent by the imaging device and displaying images according to the main image data stream.
10. The system of claim 9, wherein the receiving device is further configured to:
sending a sub-map requirement instruction to the imaging equipment;
and receiving a secondary image data stream sent by an imaging device, and performing bit operation on the data of each pixel of each frame of image in the main image data stream and the secondary image data stream to obtain an image data stream.
11. A vehicle characterized by comprising the imaging apparatus and the receiving apparatus according to any one of claims 5 to 8;
and the receiving device is used for receiving the main image data stream sent by the imaging device and displaying images according to the main image data stream.
12. The vehicle of claim 11, wherein the receiving device is further configured to:
sending a sub-map requirement instruction to the imaging equipment;
and receiving a secondary image data stream sent by an imaging device, and performing bit operation on the data of each pixel of each frame of image in the main image data stream and the secondary image data stream to obtain an image data stream.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811362126.8A CN111193901B (en) | 2018-11-15 | 2018-11-15 | Image transmission method, imaging device, system and vehicle |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811362126.8A CN111193901B (en) | 2018-11-15 | 2018-11-15 | Image transmission method, imaging device, system and vehicle |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111193901A CN111193901A (en) | 2020-05-22 |
| CN111193901B true CN111193901B (en) | 2021-11-23 |
Family
ID=70708891
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811362126.8A Active CN111193901B (en) | 2018-11-15 | 2018-11-15 | Image transmission method, imaging device, system and vehicle |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111193901B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113747046B (en) * | 2020-05-29 | 2023-06-06 | Oppo广东移动通信有限公司 | Image processing method, device, storage medium and electronic equipment |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1438612A (en) * | 1995-02-03 | 2003-08-27 | 株式会社东芝 | Image information coding/decoding system |
| CN1506971A (en) * | 2002-12-09 | 2004-06-23 | ������������ʽ���� | Semiconductor device, image data processing apparatus and method |
| US20140328538A1 (en) * | 2013-05-03 | 2014-11-06 | Samsung Electronics Co., Ltd. | Image signal processing device and method and image processing system using the same |
| CN106791858A (en) * | 2017-01-09 | 2017-05-31 | 国家新闻出版广电总局电影数字节目管理中心 | A kind of high sampling rate rgb video data lossless compression method |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2008304763A (en) * | 2007-06-08 | 2008-12-18 | Hitachi Displays Ltd | Display device |
| JP6031745B2 (en) * | 2011-10-17 | 2016-11-24 | ソニー株式会社 | Transmitting apparatus, transmitting method, and receiving apparatus |
-
2018
- 2018-11-15 CN CN201811362126.8A patent/CN111193901B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1438612A (en) * | 1995-02-03 | 2003-08-27 | 株式会社东芝 | Image information coding/decoding system |
| CN1506971A (en) * | 2002-12-09 | 2004-06-23 | ������������ʽ���� | Semiconductor device, image data processing apparatus and method |
| US20140328538A1 (en) * | 2013-05-03 | 2014-11-06 | Samsung Electronics Co., Ltd. | Image signal processing device and method and image processing system using the same |
| CN106791858A (en) * | 2017-01-09 | 2017-05-31 | 国家新闻出版广电总局电影数字节目管理中心 | A kind of high sampling rate rgb video data lossless compression method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111193901A (en) | 2020-05-22 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10110936B2 (en) | Web-based live broadcast | |
| CN104902193B (en) | A kind of method for being split processing and display to image data based on FPGA | |
| US11042770B2 (en) | Artificial intelligence based image data processing method and image sensor | |
| CN105993177A (en) | Image capture accelerator | |
| US20160381338A1 (en) | System and method for storing image data in parallel in a camera system | |
| US11800248B2 (en) | Apparatus for encoding image, apparatus for decoding image and image sensor | |
| CN103098077A (en) | Real-time video frame pre-processing hardware | |
| US10784892B1 (en) | High throughput hardware unit providing efficient lossless data compression in convolution neural networks | |
| CN109635803A (en) | Image processing method and equipment based on artificial intelligence | |
| US11856320B2 (en) | Image encoder, an image sensing device, and an operating method of the image encoder | |
| CN113946301B (en) | Tiled display system and image processing method thereof | |
| CN111193901B (en) | Image transmission method, imaging device, system and vehicle | |
| US10304213B2 (en) | Near lossless compression scheme and system for processing high dynamic range (HDR) images | |
| CN108174084A (en) | panoramic video processing method and terminal device | |
| WO2023010755A1 (en) | Hdr video conversion method and apparatus, and device and computer storage medium | |
| JP2004312073A (en) | Image processing device and camera thereof | |
| US11302035B2 (en) | Processing images using hybrid infinite impulse response (TTR) and finite impulse response (FIR) convolution block | |
| US8482438B2 (en) | Data processing device and data processing method | |
| CN114766030A (en) | Method and apparatus for improved object detection | |
| US9001267B2 (en) | Method and apparatus for tightly coupled, low power image processing | |
| CN109688314B (en) | Camera system and method with low delay, less cache and controllable data output mode | |
| CN115967784A (en) | Image transmission processing system and method based on MIPI CSI-PHY protocol | |
| US10917655B2 (en) | Video data processing using an image signatures algorithm to reduce data for visually similar regions | |
| KR101937718B1 (en) | Image Processing Method and Apparatus Using Bus Protocol Based Valid Pixel | |
| CN110475044A (en) | Image transfer method and device, electronic equipment, computer readable storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |