TWI482119B - Control method for image processing and apparatus therefor - Google Patents

Description

Image processing control method and device thereof

The present invention relates to an image processing method and apparatus thereof, and more particularly to a method and apparatus for controlling image data.

With the rapid development of technology, many consumer electronic products have evolved from traditional two-dimensional (2 Dimension, 2D) imaging to three-dimensional (3 Dimension, 3D) imaging to meet consumers' higher visual sense requirements. Electronic products with 3D imaging capabilities have emerged on the market, such as 3D TVs and 3D mobile phones. It is expected that electronic products with 3D imaging capabilities will be one of the future trends. Electronic products with 3D imaging capabilities must also be matched with 3D video content. The 3D video content can be provided by, for example, an electronic device having a 3D imaging capability, such as a 3D digital camera.

Making 3D images can be combined into a single file using two different perspective images in a specific format, such as Multi-Picture Object (MPO). In the case of using a 3D digital camera to generate 3D images, two camera lenses can be used to capture two images of different viewing angles, and the captured images are compressed and combined into a single file. Since it is necessary to perform compression processing on two images having different viewing angles, if the two images are sequentially processed and compressed, there may be a problem of low performance. A more time-saving method is to perform image processing and compression on the two images in parallel using two sets of processing units. However, the use of two sets of processing units for image processing and compression at the same time must consider the synchronization between the two processing units and the timing of the final merge. Therefore, how to efficiently produce 3D image files is indeed an important issue in the current field.

The invention provides an image processing control method and a device thereof, which process and compress two images with different viewing angles in parallel and combine them into a single file by data transmission between processing units.

The present invention provides an image processing control method, which is applicable to the first image processing unit and the second image processing unit, and includes the following steps. First, the first image processing unit and the second image processing unit process the main image and the auxiliary image in parallel to respectively generate a main image file and an auxiliary image file, wherein the first image processing unit simultaneously writes the main image file when the main image file is generated. Image file to an external storage unit. Then, after the second image processing unit compresses and generates the auxiliary image file, the transfer completion notification is sent to the first image processing unit. Then, after receiving the completion notification, the first image processing unit reads the auxiliary image file, writes the auxiliary image file to the external storage unit, and attaches the auxiliary image file to the main image file in a file format to obtain the multi-object image file. Finally, the first image processing unit updates the header of the multi-object image file.

The present invention also provides an image processing apparatus including a first image processing unit and a second image processing unit. The first image processing unit generates a main image file by using the main image compression, wherein the first image processing unit simultaneously writes the main image file to the external storage unit when the main image file is generated. The second image processing unit is coupled to the first image processing unit. When the first image unit compresses and generates the main image file, the auxiliary image is processed in parallel to compress and generate the auxiliary image file, and after the compressed image file is generated, the notification is transmitted to the second image processing unit. The first image unit. After receiving the completion notification, the first image processing unit reads the auxiliary image file from the second image processing unit, writes the auxiliary image file to the external storage unit to be attached to the main image file to obtain the multi-object image file, and updates the multi-object image file. Head.

Based on the above, the present invention provides an image processing control method and apparatus thereof, such that the first and second image processing units respectively process the compressed main image file and the auxiliary image file. The first image processing unit simultaneously writes the main image file to the external storage unit when compressing and generating the main image file, and then attaches the auxiliary image file to obtain the multi-image object file and updates the header content, and the flow control method can be used to The simplified and time-saving process results in a multi-image object file.

The above described features and advantages of the present invention will be more apparent from the following description.

FIG. 1 is a flow chart showing the steps of an image processing control method according to an embodiment of the invention. The method is applicable to a first image processing unit and a second image processing unit. Referring to FIG. 1, first, in step S101, the first image processing unit and the second image processing unit process a main image and an auxiliary image in parallel to respectively compress and generate the main image file and the auxiliary image file. The first image processing unit simultaneously writes the main image file to an external storage unit when generating the main image file. It is worth mentioning that the main image and the attached image are compressed into the main image file and the attached image file in the file format of the Joint Photographic Experts Group (JPEG) or other commonly used image file format, but the present invention Not limited to the above format.

Next, in step S102, the second image processing unit compresses the transmission completion notification to the first image processing unit after generating the auxiliary image file. Then, in step S103, after receiving the completion notification, the first image processing unit reads the auxiliary image file from the second image processing unit, and writes the auxiliary image file to the external storage unit, and attaches the auxiliary image file to the main file file format. Image file to get a multi-object image file. The file format can be a multi-image object file (MPO) format, but the format is not limited. Finally, in step S104, the first image processing unit updates the header of the multi-object image file to complete the creation of the multi-object image file.

FIG. 2 is a partial flowchart of a method for controlling image processing according to an embodiment of the present invention, showing an action of the image processing control method before performing the process shown in FIG. 1 in an embodiment, and can be regarded as utilizing The pre-set action required to compress the main image and the attached image in JPEG format or other common image formats. Referring to FIG. 2, first, in step S201, the first image capturing unit and the second image capturing unit respectively capture the main image and the auxiliary image, wherein the first image capturing unit and the second image capturing unit are respectively coupled The first image processing unit and the second image processing unit are connected. Then, in step S202, the first image processing unit and the second image processing unit respectively receive the main image and the auxiliary image from the first image capturing unit and the second image capturing unit.

In step S203, the first image processing unit and the second image processing unit respectively perform preprocessing procedures on the main image and the auxiliary image. The pre-processing program includes the following steps: obtaining information related to image capture, setting image processing parameters, and performing image processing such as image correction to the first image capturing unit and the second image capturing unit, but the present invention is not limited to the above. . Then, in step S204, the first image processing unit and the second image processing unit respectively create a header of the main image and the auxiliary image according to the main image and the auxiliary image, wherein the header is used to record the attribute information and the shooting data of the image. The content is attached to the image file using the Exchangeable Image Format (EXIF) or other general-purpose format.

In step S205, the first image processing unit and the second image processing unit respectively generate a main thumbnail (Thumbnail) and an affiliate thumbnail using the main image and the auxiliary image. The main thumbnail image and the attached thumbnail image are the main image and the attached image of the reduced-size version, which are used to identify the image content during file management, and can be obtained by down sampling and then compressing the main image and the auxiliary image. . In addition, the size of the thumbnail is different from that of the embodiment. In the present embodiment, the size of the main thumbnail and the attached thumbnail is 160*160 pixels, and the compression processing is performed in the JPEG format.

In step S206, the first image processing unit and the second image processing unit respectively generate a main thumbnail image and an auxiliary thumbnail image by using the main image and the auxiliary image, wherein the main thumbnail image and the auxiliary thumbnail image generated in this step are larger than the main thumbnail image. The thumbnail and the attached thumbnails are mainly used for display in a display unit of an electronic device such as a digital camera or a personal computer. The main thumbnail image and the attached thumbnail image can also be obtained by downsampling the main image and the auxiliary image and then performing compression processing. In this embodiment, the main thumbnail image and the attached thumbnail image are in accordance with the Video Graphics Array (VGA) standard, that is, the image of 640*480 pixels, which is the same as the above-mentioned main thumbnail image and the auxiliary thumbnail image, and The compression processing is performed in the JPEG format, but the present invention is not limited to the above.

FIG. 3 is a flow chart showing the steps of simultaneously writing a main image file to an external storage unit when the first image processing unit generates a main image file according to an embodiment of the present invention, corresponding to the first image processing in step S101 of FIG. An embodiment of the operation of the unit. Referring to FIG. 3, first, in step S1011, the first image processing unit temporarily stores the header of the main image file and the content of the compressed main image file in the first memory unit in a first-in-first-out manner, where the first The memory unit is coupled to the first image processing unit. In other words, the first image processing unit temporarily stores the completed portion in the first memory unit while compressing and generating the main image file.

In step S1012, the first image processing unit reads the temporary image in the first memory unit in a preset divided size batch, and compresses the generated main image file to write to the external storage unit. The external storage unit has a storage space of a larger first memory unit, and may be a portable storage unit, such as a memory card of a commonly used format, an external memory, etc., and the present invention is not limited to the above.

The size of the preset division is set depending on the actual implementation, such as the size of the first memory unit, the compression processing speed, the external storage unit write speed, and the like. In an embodiment, the preset partition size is 128 kilobytes (K byte), and the preset is when the first memory unit is filled with a predetermined segmentation, that is, 128K of the main image file content. The main image file of the split size is written to the external storage unit. Finally, in step S1013, after all the main image files have been written into the external storage unit, the header of the main image file is written into the header of the main image file in the external storage unit.

FIG. 4 illustrates an embodiment of steps S102 through S104 of FIG. 1 in accordance with an embodiment of the present invention. Referring to FIG. 4, first, in step S1021, the second image processing unit temporarily stores the compressed auxiliary image file in the second memory unit in a first-in first-out manner, and transmits the completion notification to the first image processing after completion. unit. The completion notification includes information about a memory location of the auxiliary video file in the second memory unit and a memory size. Then, in step S1031, the first image processing unit can read the auxiliary image file from the second memory unit according to the information in the completion notification, and temporarily store the auxiliary image file in the first memory unit.

In step S1032, the first image processing unit reads the auxiliary image file from the first memory unit and writes it to the external storage unit. Note that the auxiliary image file merges the main image file in an additional manner to write to the external storage unit. Then, in step S1041, information such as the memory start position of the attached image file and the file size of the attached image file in the multi-object image file are written into the header of the multi-object image file to complete the multi-object image file. The establishment of.

After the image processing control method of the present invention is described with reference to the drawings in the above embodiments, an image processing apparatus using the control method will be described below. FIG. 5 is a block diagram of an apparatus for processing an image processing apparatus according to an embodiment of the invention. Referring to FIG. 5 , the image processing apparatus 300 includes a first image processing unit 320 and a second image processing unit 330 , and is externally connected to an external storage unit 370 . The first image processing unit 320 generates a main image file MF by using the main image MIMG compression. The first image processing unit 320 simultaneously writes the main image file MF to the external storage unit 370 when the main image file MF is generated.

The second image processing unit 330 is coupled to the first image processing unit 320. When the first image unit 320 compresses and generates the main image file MF, the auxiliary image SIMG is processed in parallel to compress and generate the auxiliary image file SF. The second image processing unit 330 transmits the completion notification NF to the first image unit 320 after the auxiliary image file SF is compressed. After receiving the completion notification NF, the first image processing unit 320 reads the auxiliary image file SF, writes the auxiliary image file SF to the external storage unit 370, and attaches the auxiliary image file SF to the main image file MF in a file format such as an MPO format. Obtain a multi-object image file and update the header of the multi-object image file. For other detailed implementation manners, reference may be made to the above embodiments, and details are not described herein again.

FIG. 6 is a block diagram of an image processing apparatus according to an embodiment of the present invention. Compared with the embodiment shown in FIG. 5, FIG. 6 provides a more detailed implementation of the image processing apparatus. Referring to FIG. 6 , the image processing apparatus 300 includes a first image capturing unit 310 , a second image capturing unit 312 , a first image processing unit 320 , a second image processing unit 330 , a first memory unit 340 , and a second memory unit . 350.

The first image processing unit 320 and the second image processing unit 330 further include a main image processor 322 and an auxiliary image processor 332, sensor interfaces 323 and 333, and a general purpose input/output (GPIO) receiving interface. 326 and universal input/output transmission interface 336, secure digital input/output (Secure Digital I/O, SDIO) receiving interface 327 and secure digital input/output transmission interface 337. The first image processing unit 320 is further connected to the external display unit 260 and the external storage unit 270 through the SDIO interface 328.

The first image capturing unit 310 and the second image capturing unit 312 are respectively coupled to the first image processing unit 320 and the second image processing unit 330, and respectively capture the main image MIMG and the auxiliary image SIMG, and then transmit the image to the first image processing. Unit 320 and second image processing unit 330. The first image processing unit 320 and the second image processing unit 330 receive the main image MIMG and the auxiliary image SIMG through the sensor interfaces 323 and 333, respectively, and use the main image processor 322 and the auxiliary image processor 332 to view the main image MIMG and The attached image SIMG performs image processing. The operation of the image processing flow corresponding to the method of FIGS. 1 to 4 by the image processing apparatus 300 will be described in detail below with reference to the drawings.

FIG. 7 is a schematic diagram showing the task operation timing of the image processing apparatus according to an embodiment of the invention. Referring to FIG. 6 and FIG. 7 , first, the first image processing unit 320 and the second image processing unit 330 respectively process the task t1 and the task t5 by using the main image processor 322 and the auxiliary image processor 332 to respectively compress and generate the main image file. The MF and the attached video file SF, and the compressed portions are temporarily stored in the first memory unit 340 and the second memory unit 350, respectively. It should be noted that the first image processing unit 320 and the second image processing unit 330 have completed the pre-action shown in the embodiment of FIG. 2 on the main image MIMG and the auxiliary image SIMG before executing the task t1 and the task t5.

After the first image processing unit 320 starts executing the task t1 to compress and generate the main image file MF, the first image processing unit 320 starts a waiting program t3 by using the task t2, and continues to wait for receiving the GPIO receiving interface 326 in the waiting program t3. The completion notification NF transmitted by the second image processing unit 330. On the other hand, when the first image processing unit 320 performs the task t1, the amount of data of the compressed main image file MF is up to a predetermined segment size (for example, 128K byte set in the foregoing embodiment), the first image The processing unit 320 writes the compressed main image file MF temporarily stored in the first memory unit 340 to the external storage unit 370 through the SDIO interface 328 in a predetermined split size at tasks t41 to t4n. The file size of the main image file MF is N times of the preset cut size, and the task t4n indicates that the last split of the main image file MF has been preset by the split size.

After the second image processing unit 330 compresses the auxiliary image file SF by using the auxiliary image processor 332, the second image processing unit 330 transmits the completion notification NF to the first image processing unit 320 through the GPIO transmission interface 336, wherein the completion notification NF includes The auxiliary image file SF is temporarily stored in the memory size and position in the second memory unit 350. Next, the first image processing unit 320 receives the completion notification NF by using the GPIO receiving interface 326, and the task t3 returns the information of the completion notification NF to the main image processor 322.

The main image processor 322 terminates the task t3 upon receiving the completion notification, and if the first memory unit 340 has sufficient space, the second image processing unit 330 is required to return the auxiliary image file SF and temporarily store the content according to the content of the completion notification. In the first memory unit 340. It should be noted that the space in the first memory unit 340 for temporarily storing the main image file MF can be reused, and when the first memory unit 340 has sufficient space, it can also be cut into several smaller split batch transmissions. The embodiment is adjusted according to actual conditions, and is not limited to the above method.

After the first image processing unit 320 completes the task t4n, that is, after the complete writing of the main image file MF, the main image processor 322 starts the task t7, and the auxiliary image file SF temporarily stored in the first memory unit 340 is attached. The main image file MF is written to the external storage unit 270 in the MPO file format through the SDIO interface 328. Finally, the first image processing unit 320 starts the task t8 by using the main image processor 322, updates the header of the merged MPO file, and writes the memory starting position and the file size of the attached auxiliary image file in the header. . Finally, the file content stored in the external storage unit 270 can be referred to the description of FIG. 8 below.

FIG. 8 is a schematic diagram showing data of a multi-image object file according to an embodiment of the invention. Referring to FIG. 8, the multi-image object file 80 includes a header data content 810, a main image content 820, an auxiliary image header content 830, and an affiliate image content 840. In other words, the multi-image object file 80 includes a main image file. And the attached image file is attached to the main image file content in an additional manner. The header 810 of the multi-image object file 80 is the header of the main image file plus the following auxiliary image header 830 and the attached image content 840 respectively, and the memory location and the memory size are respectively transmitted. When other display devices perform three-dimensional image display, the display device can successfully distinguish the content information of the two images.

It is worth mentioning that the main thumbnail 811 and the main thumbnail 812, as well as the attached thumbnail 831 and the attached thumbnail 832 are originally stored in their respective headers, that is, the header 810 and the attached image archive header 830. In the case where the display device is to use these thumbnails, the information accessed in the header 810 can also be directly utilized.

In summary, the present invention provides an image processing control method and apparatus thereof, which can respectively receive two images with different viewing angles by using two sets of image processing units, and compress and process two images in parallel, and then pass between two sets of image processing units. Workflow control and data transfer, the time required to create a multi-image object file can be greatly shortened without the need for additional hardware or enhanced computing power, increasing the availability and convenience of the product, and At the same time, it satisfies the needs of consumers who want to watch and watch.

Although the present invention has been disclosed in the above embodiments, it is not intended to limit the invention, and any one of ordinary skill in the art can make some modifications and refinements without departing from the spirit and scope of the invention. The scope of the invention is defined by the scope of the appended claims.

S101 to S104, S201 to S206, S1011 to S1013, and S1021 to S1041. . . step

300. . . Image processing device

310, 312. . . Image capture unit

320, 330. . . Image processing unit

323, 333. . . Sensor interface

322. . . Primary image processor

332. . . Affiliated image processor

326. . . GPIO receiving interface

336. . . GPIO transmission interface

327. . . SDIO receiving interface

337. . . SDIO transmission interface

328. . . SDIO interface

340, 350. . . Memory unit

370. . . External storage unit

MIMG. . . Main image

SIMG. . . Affiliated image

NF. . . Completion notice

MF. . . Primary image file

SF. . . Affiliated image file

T1~t3, t41~t4n, t5~t8. . . Processor task program

FIG. 1 is a flow chart showing the steps of an image processing control method according to an embodiment of the invention.

FIG. 2 is a partial flow chart of a method for controlling image processing according to an embodiment of the invention.

FIG. 3 is a flow chart showing the steps of simultaneously writing a main image file to an external storage unit when the first image processing unit generates a main image file according to an embodiment of the invention.

FIG. 4 illustrates an embodiment of steps S102 through S104 of FIG. 1 in accordance with an embodiment of the present invention.

FIG. 5 is a block diagram of an apparatus for processing an image processing apparatus according to an embodiment of the invention.

FIG. 6 is a block diagram of an apparatus for processing an image processing apparatus according to an embodiment of the invention.

FIG. 7 is a schematic diagram showing the task operation timing of the image processing apparatus according to an embodiment of the invention.

FIG. 8 is a schematic diagram showing data of a multi-image object file according to an embodiment of the invention.

S101～S104. . . step

Claims (14)

The image processing control method is applicable to a first image processing unit and a second image processing unit. The first image processing unit and the second image processing unit process a main image and an auxiliary image in parallel to respectively generate a compression image. a main image file and an attached image file, wherein the first image processing unit simultaneously writes the main image file to an external storage unit when the main image file is generated, wherein the main image and the auxiliary image have different viewing angles After the second image processing unit compresses and generates the auxiliary image file, transmitting a completion notification to the first image processing unit; after receiving the completion notification, the first image processing unit reads the auxiliary image file Writing the auxiliary image file to the external storage unit, and attaching the auxiliary image file to the main image file in a file format to obtain a multi-object image file; and the first image processing unit updates the multi-object image file a header. The method of claim 1, wherein before the step of separately processing the main image data and the auxiliary image data to separately generate the main image file and the auxiliary image file, the method further comprises: utilizing a first image The first image capturing unit and the second image capturing unit are respectively coupled to the first image processing unit and the second image capturing unit and the second image capturing unit respectively Two image processing units; Receiving a primary image and the auxiliary image; performing a pre-processing procedure on the primary image and the auxiliary image; respectively forming a header of the primary image and a header of the auxiliary image according to the primary image and the auxiliary image; A main thumbnail image and an auxiliary thumbnail image are respectively generated by using the main image and the auxiliary image; and a main thumbnail image and an auxiliary thumbnail image are respectively generated by using the main image and the auxiliary image. The method of claim 2, wherein the step of writing the main image file to the external storage unit while the first image processing unit compresses the main image file comprises: The header and the content of the main image file that has been compressed are temporarily stored in a first memory unit in a first-in-first-out manner; and the batch is read and stored in the first memory unit by a preset divided size. The main image file that has been compressed is written into the external storage unit; and after the main image file is all written into the external storage unit, the header of the main image file is written into the main image in the external storage unit. The header of the file. The method of claim 3, wherein the first image processing unit and the second image processing unit respectively process the main image and the auxiliary image in parallel to respectively compress and generate the main image file and the auxiliary image file. The steps include: The auxiliary image file that has been compressed is temporarily stored in a second memory unit in a first in first out manner. The method of claim 4, wherein the completion notification includes a memory location and a memory size corresponding to the auxiliary image file in the second memory unit. The method of claim 5, wherein the first image processing unit reads the auxiliary image file from the second image processing unit after receiving the completion notification, and the step of writing the external storage unit includes: An image processing unit reads the auxiliary image file from the second memory unit according to the completion notification, and temporarily stores the image file in the first memory unit; and the first image processing unit reads the auxiliary image file from the first memory unit Write to the external storage unit. The method of claim 6, wherein updating the header of the multi-object image file comprises: a memory starting position of the multi-object image file corresponding to the auxiliary image file and the auxiliary image file The file size is written into the header of the multi-object image file. An image processing apparatus includes: a first image processing unit that generates a main image file by using a main image compression, wherein the first image processing unit simultaneously writes the main image file to an external storage when the main image file is generated a second image processing unit coupled to the first image processing unit, and parallel processing one when the first image unit compresses and generates the main image file The auxiliary image is compressed to generate an auxiliary image file, and after the compressed image file is generated, a completion notification is transmitted to the first image unit, wherein the main image and the auxiliary image are two images having different viewing angles; After receiving the completion notification, an image processing unit reads the auxiliary image file, writes the auxiliary image file to the external storage unit, and attaches the auxiliary image file to the main image file in a file format to obtain a multi-object image. File and update the header of the multi-object image file. The device of claim 8, wherein the device further comprises: a first image capturing unit and a second image capturing unit, respectively coupled to the first image processing unit and the second image processing unit And capturing the main image and the auxiliary image respectively; and the first image processing unit and the second image processing unit respectively receiving the main image and the auxiliary image, respectively performing preprocessing on the main image and the auxiliary image a program, respectively, a header of the main image and a header of the auxiliary image, and respectively using the main image and the auxiliary image to generate a main thumbnail and a main thumbnail, and an auxiliary thumbnail and an auxiliary thumbnail . The device of claim 9, wherein the device further comprises: a first memory unit coupled to the first image processing unit for temporarily storing the main image generated by the first image unit compression in a first-in first-out manner Storing the first image unit and reading the main image file temporarily stored in the first memory unit in a predetermined divided size batch for writing to the external storage The unit, when the primary image file is completely written into the external storage unit, the first memory unit writes the header of the primary image file into the header of the primary image file in the external storage unit. The device of claim 10, wherein the device further comprises: a second memory unit coupled to the second image processing unit for temporarily storing the second image unit compressed by the second image unit in a first-in first-out manner Affiliated image file. The device of claim 11, wherein the completion notification includes a memory location and a memory size corresponding to the auxiliary video file in the second memory unit. The device of claim 12, wherein: the first image processing unit reads the auxiliary image file from the second memory unit according to the completion notification, and temporarily stores the image file in the first memory unit, and then from the The first memory unit reads the auxiliary image file and writes the external image storage unit. The device of claim 13, wherein the first image processing unit writes a memory starting position of the multi-object image file corresponding to the auxiliary image file and the file size of the auxiliary image file. The header of the multi-object image file updates the header of the multi-object image file.