JP2018182551A - Imaging apparatus - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an imaging apparatus capable of shortening a display delay time, even when an image area is divided for performing image processing by using a plurality of image processors.SOLUTION: The imaging apparatus includes an imaging part, a division part which divides captured image data, first and second image processing devices which image-process divided division image data, an image data transfer part which transfers image data between the first and second image processing devices, and a display part for displaying image-processed image data. The first image processing device includes a memory for display which stores synthetic image data obtained by synthesizing first image data generated from first division image data and second image data transferred by data transfer means as display data and a display control part which reads out the display data from the memory for display and displays it on the display part. The division position of the division part is set according to the processing of the display control part and the throughputs of the first and second image processing devices are decided.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an imaging apparatus that divides an input image and processes the image.

  In recent years, cameras have been further enhanced in resolution to 4K (3840 × 2160 pixels). In such an apparatus, when the image processing capability is insufficient with only one image processor (image processing LSI), there is a technology for cutting out a plurality of divided images from one input image and performing image processing simultaneously with the plurality of image processors. It is used. When performing image processing by dividing an input image into a plurality of pieces, there is a technology in which image processing is performed by overlapping a part of images between adjacent divided images so that continuity of the image is not lost at the boundary portion. It is disclosed.

  In Patent Document 1, when cutting out a plurality of divided images from an input image, an overlapping area is provided for divided images adjacent to each other, and image processing is performed including the overlapping area. Further, in Patent Document 2, when one image is divided and processed by a plurality of image processing processors, the processing amount for each image processing processor is determined according to the processing time and data transfer amount of each image processing processor. By doing this, the processing load of each image processing processor is distributed.

  In recent years, during high-resolution moving image shooting, it is desired to display shooting data on an external display capable of displaying high-resolution images as well as the low-resolution display provided on the camera body. The main application of the display on this display medium is to confirm the influence on the recorded image with respect to the photographing operation such as the focus position which is difficult to determine with a low resolution monitor. Therefore, real-time property is also important for the display on the external display, and it is desirable that the camera also has a short delay time from imaging to display (hereinafter, display delay time).

JP, 2011-97474, A JP, 2016-10112, A

  However, as in the above-mentioned patent document 1, image processing processors are connected by the number according to the target performance, and in a system capable of extending the image processing performance by processing an image divided into areas, divided and generated recordings It is necessary to transfer image and display data between image processing apparatuses. Therefore, the display delay time is increased due to queuing of a plurality of image processing apparatuses and transfer processing as compared with the case of processing by a single image processing apparatus.

  Further, although the processing load of a plurality of image processors can be distributed in the above-mentioned Patent Document 2, the display delay time increases as the processing load increases.

  The problem of the display delay time when one image is divided and processed using a plurality of image processing processors will be described using FIG. FIG. 6 is a simplified diagram showing processing timings when processing is performed by a plurality of image processing processors, which is a problem to be solved by the present invention.

  FIG. 6 shows processing timing in the case where image processing is performed by dividing the image data into upper and lower portions by using two image processing processors after acquisition of imaging data, and T1 indicates acquisition timing of imaging data. . T2 and T5 indicate timings of performing upper and lower half image processing in each image processing processor by cutting out upper and lower halves from the imaging data. T3 and T6 indicate the timing when each image processing processor has transferred the other half of the imaging data. T4 and T7 indicate the display timing on the display medium. Therefore, D1 and D2 from acquisition of imaging data to display indicate display delay time. It can be seen from FIG. 6 that the display delay time of D1 is longer than that of D2. When the display is performed from the first image processor, the display delay time is increased by waiting for the transfer process. When display is performed from the second image processing apparatus, queuing of transfer processing does not occur, but when display is performed on a plurality of display media, there is a problem that the processing load of one image processing processor becomes high.

  Therefore, the present invention has been made to solve the above-described problems, and can reduce display delay time even when image processing is performed by dividing an image area using a plurality of image processing processors. It aims at providing an imaging device.

In order to achieve the above object, an image processing apparatus according to the present invention is
An imaging unit that acquires captured image data at an arbitrary frame rate;
A division unit that divides captured image data;
First and second image processing apparatuses that perform image processing on divided image data divided by the dividing unit;
An image data transfer unit for transferring image data between the first and second image processing apparatuses; and a display unit for displaying image data subjected to image processing,
The first image processing device
A display memory for combining the first image data generated from the first divided image data and the second image data transferred by the data transfer means, and storing it as display data;
A display control unit which reads display data from the display memory and displays it on a display unit;
Equipped with
The division position of the division unit is set according to the processing of the display control unit, and the processing amounts of the first and second image processing apparatuses are determined.

  According to the image processing apparatus according to the present invention, when an input image is divided and image processing is performed by a plurality of image processing processors, display delay time due to image transfer between the image processing processors can be minimized. It becomes.

Configuration diagram of imaging apparatus in the present embodiment Schematic of division processing of a captured image Flowchart of Display Processing of Displayed Image on Display Device in this Embodiment Diagram showing the processing range of the first and second image processing processors in the present embodiment The figure which shows the processing timing of the display process to the display device of the captured image in a present Example. Diagram showing processing timing that is the subject of display delay amount of captured image

  Hereinafter, preferred embodiments of the present invention will be described with reference to the drawings. First, FIG. 1 is a block diagram showing an example of the configuration of an imaging apparatus according to this embodiment.

  Reference numerals 100a and 100b respectively process and output divided and input image data by the first image processor and the second image processor.

  111 is an operation unit. It is configured by a recording start / end button, a touch panel, and the like provided on the exterior portion of the imaging apparatus main body, and transmits the user's operation to the first image processor 100a.

  Reference numeral 120 denotes an imaging optical system. A subject image is formed on the imaging surface of the imaging element 141 via an optical lens group such as the variable magnification optical system 121.

  Reference numeral 141 denotes an imaging device. A subject image formed on an imaging surface is photoelectrically converted to generate an imaging signal, and image data obtained by A / D conversion is output.

  142 is a division part. Based on the control of the first image processor 100a, the processing ranges of the first image processor 100a and the second image processor 100b are cut out from the image data output from the imaging device 141 and output.

  Reference numeral 151 denotes a division information generation unit. The amount of display data output from the second image processor 100b is calculated, and based on the amount of data, the first image processing is performed on the division information indicating the processing range in each of the image processors 100a and 100b of the division unit 142 It outputs to the processor 100a.

  161 and 162 are display devices. Display data output from the second image processor 100b is displayed.

  171 is a recording medium. Image data for recording is recorded by the first image processor 100a.

  Next, the internal configuration of the first image processor 100a will be described. Reference numeral 101a denotes a control microcomputer, which includes a non-volatile memory and a work memory, and reads and writes temporary data to the work memory based on programs and data stored in the non-volatile memory. Take control. 102a is a control bus. 103 a is a communication unit. The control microcomputer 101a communicates control information with the second image processor 100b via the communication unit 103a.

  Reference numeral 104a denotes an image processing unit. A correction process and a development process are performed on the image data output from the dividing unit 142. It also performs resizing to an image size that matches the display device. Reference numeral 106a denotes a data transfer unit. It receives the image data output from the second image processor 100b or transmits the image data processed by the image processor 104a to the second image processor 100b. Reference numeral 105a denotes an image memory. The image data processed by the signal processing unit 104 a and the image data of the second image processing processor 100 b transferred using the data transfer unit 106 a are stored.

  Reference numeral 109a denotes a recording control unit. According to the recording start / end instruction from the control microcomputer 101a, the image data output from the image processing unit 104a and the image data transferred via the data transfer unit 106a are encoded, and the recording image data is recorded on the recording medium Record at 171.

  Next, an internal configuration of the second image processor 100b will be described. The configuration of the second image processor 100b is the same as that of the first image processor 100a, but the processing content borne by some blocks is different in the system configuration.

  The processing content of each block 101b to 106b is equivalent to that of each block 101a to 106a of the first image processor 100a. Reference numerals 107a and 108a denote display control units. Based on the instruction of the control microcomputer 101a, the display image data output from the image processing unit 104a is superimposed on display information composed of characters and icons, and the display image data is output to the display devices 161 and 171.

  Next, division processing of a captured image will be described with reference to FIG. In the imaging apparatus of the present embodiment, at the time of capturing a moving image, the captured image is divided into two up and down, and the two image processors 100 a and 100 b process each of the divided images.

  FIG. 2A is an example of a captured image of one frame captured by the imaging element 141. Here, the resolution of the captured image is set to horizontal 3840 × vertical 2160 pixels, and the imaging frame rate is set to 60 fps.

  FIG. 2B shows a divided image 211 and a divided image 212 obtained by dividing the captured image into upper and lower portions by the dividing unit 142. In the imaging apparatus of this embodiment, the captured image is divided into upper and lower parts, and the upper half is input to the first image processor 100a and the lower half is input to the second image processor 100b. Since the image processors 100a and 100b individually perform image processing, each divided image is provided with an overlap region in an adjacent portion so that continuity of the image is not lost at the boundary portion.

  FIG. 2C shows a recording image 221 and a recording image 222 which the image processors 100a and 100b process and output the divided images 211 and 212, respectively. The overlap area of the divided images 211 and 212 is removed by image processing in the image processing processors 100a and 100b, and the recording images 221 and 222 are output without the overlap area.

  FIG. 2D shows display images 230 and 240 displayed on the display devices 161 and 162. By joining the recording images 221 and 222, the display devices 161 and 162 display the images while maintaining the continuity of the boundary portion. The display image 230 outputs the recorded image to the external display device 161 as it is without resizing. The display image 240 is an image which has been subjected to reduction processing in the image processing units 104 a and 104 b. Here, the recording image of horizontal 3840 × vertical 2160 pixels is resized by half both horizontally and vertically, and horizontal 1920 × vertical 1080 It is an image of pixels.

  Next, the process of displaying the captured image of the present embodiment on the display device will be described with reference to FIGS. 3, 4 and 5. FIG. 3 is a flowchart of a process of displaying a captured image of the present embodiment on a display device. FIG. 4 is a diagram showing the processing range of the first and second image processing processors in the present embodiment. FIG. 5 is a diagram showing processing timing of display processing of the captured image of the present embodiment on the display device.

  The control flow of this embodiment will be described in detail with reference to FIG.

  In step S301, based on an instruction of the control microcomputer 101a, acquisition of captured image data captured by the imaging element 141 is performed.

  In step S302, the division information generation unit 151 calculates the amount of display data to be displayed on the display device. It is possible to switch the presence / absence of output to the display device by instructing from the user using the operation unit 111. In addition, it is possible to automatically detect the presence or absence of connection between the imaging device and an external display device and switch the presence or absence of display. The division information generation unit 151 detects the presence or absence of an output to each display device, and calculates the amount of display data.

  In step S303, the amount of display data acquired by the division information generation unit 151 is compared with a predetermined value. The predetermined value is the maximum amount of display data that can be processed when processing is performed with the first and second image processors 100a and 100b equally dividing the captured image. In this embodiment, the display data amount in the case of displaying a display image of horizontal 3840 × vertical 2160 pixels from the second image processor 100 b is set to a predetermined value. Therefore, when the display image 230 (horizontal 3840 × vertical 2160 pixels) and the display image 240 (horizontal 1920 × vertical 1080 pixels) in FIG. 2D are simultaneously displayed, the display data amount is larger than a predetermined value. When only the display image 240 (horizontal 1920 × vertical 1080 pixels) is displayed, the amount of display data is smaller than a predetermined value. If the display data amount is larger than the predetermined value, the process proceeds to step S304. If the display data amount is smaller than the predetermined value, the process proceeds to step S305.

  In step S304, the division information generation unit 151 generates division information so as to reduce the processing range of the second image processor 100b that outputs data to the display device, and the process advances to step S306.

  The generation of division information in step S304 will be described. FIG. 4A shows the processing range of the first and second image processing processors 100b when the processing range of the second image processing processor 100b is reduced. The dotted line P0 in FIG. 4A indicates the dividing position in equal division, and the line P1 indicates the dividing position in the case where the processing range of the second image processor 100b is reduced. H1 and H2 indicate the processing ranges of the first and second image processors 100a and 100b in the case of division by the line P1. Since the processing range of the second image processor 100b is reduced, the processing range H2 is smaller than the processing range H1. In step S304, the division information generation unit 151 generates division information so that the processing range H2 of the first and second image processing processors 100b is smaller than the processing range H1.

  In step S305, the division information generation unit 151 generates division information so as to increase the processing range of the second image processor 100b that outputs data to the display device, and the process advances to step S306.

  The generation of the division information in step S305 will be described. FIG. 4B shows the processing range of the first and second image processing processors 100b when the processing range of the second image processing processor 100b is increased. The line P2 in FIG. 4B indicates the division position when the processing range of the second image processor 100b is increased. H'1 and H'2 indicate the processing ranges of the first and second image processors 100a and 100b in the case of division by the line P2. Since the processing range of the second image processor 100b is increased, the processing range H'2 is larger than the processing range H'1. In step S305, the division information generation unit 151 generates division information so that the processing range H'2 of the first and second image processors 100a and 100b is larger than the processing range H'1.

  In step S306, the dividing unit 142 divides the captured image data acquired in step S301 based on the division information generated by the division information generation unit 151. The divided image 211 of FIG. 2B is processed by the first image processor 100a, and the divided image 212 is processed by the second image processor 100b.

  In step S307, the two image processors 100a and 100b perform development processing on each of the divided images. The divided images subjected to the development processing are stored in the respective image memories 105a and 105b.

  In step S308, the divided images are transferred from the data transfer units 106a and 106b and stored in the image memory 105b. The composite image is stored in the image memory 105 b using the divided image stored in step S 307 and the transferred divided image.

  In step S309, the composite image combined in step S308 is output from the display control units 107b and 108b to the display devices 161 and 171, and the display image is displayed.

  Here, the processing timing of each step will be described with reference to FIG. FIG. 5A shows the processing timing when the processing range of the second image processor 100b is reduced, and FIG. 5B shows the case when the processing range of the second image processor 100b is increased Shows the processing timing of

  First, the case where the processing range of the second image processor 100b is reduced in step S304 will be described.

  The image sensor output timing in FIG. 5A indicates the processing timing of the output of the image sensor 141 and the input to the dividing unit 142. The division unit output timing indicates the output processing timing from the division unit 142 to the first and second image processors 100a and 100b. Since the output to the first image processor 100a is the output of the upper part of the image, the processing timing is at the first half of 1 V, and the output to the second image processor 100b is the output of the lower part of the image Therefore, the processing timing is in the second half of 1V. The transfer data reception timing is a timing at which the divided image is received.

  At the transfer data reception timing of the first image processor 100a, the transfer data reception timing starts from the completion of the division output timing of the second image processor 100b. Similarly, the transfer data reception timing of the second image processor 100b becomes the reception timing of transfer data from the completion of the division output timing of the first image processor 100a. The display device input timing indicates the input timing of display data to the display device. D3 indicates a display delay time, and indicates a display delay time from shooting, that is, from readout of the imaging device 141 to start of display. When the amount of display data is larger than a predetermined value, it is possible to make the display delay time to each display device 161, 162 uniform by reducing the processing range of the second image processor 100b.

  Next, the case where the processing range of the second image processor 100b is increased in step S305 will be described.

  FIG. 4 (b) shows each processing timing as in FIG. 4 (a). Similarly, D4 indicates a display delay time. When the amount of display data is smaller than a predetermined value, it is possible to shorten the display delay time to each display device 161, 162 by increasing the processing range of the second image processor 100b.

  Thereby, when the input image is divided and image processing is performed by a plurality of image processing processors, display by image transfer is performed by switching the processing amounts of the first and second image processing processors according to the display data amount. It is possible to minimize the delay time.

  The preferred embodiment of the present invention has been described above, but the configuration of the imaging apparatus shown in FIG. 1 is an example, and the configuration of the imaging apparatus according to the present invention is a diagram if the operation of each embodiment can be performed. It is not limited to the configuration shown in 1.

  Furthermore, the present invention is not limited to these embodiments, and various modifications and changes can be made within the scope of the invention.

100a first image processor, 100b second image processor,
Control microcomputers 101a, 101b 100a, 100b,
Control buses 102a, 102b 100a, 100b,
Communication units 103a, 103b 100a, 100b,
Image processing units 104a and 104b 100a and 100b;
Image memory 105a, 105b 100a, 100b,
Data transfer units 106a, 106b 100a, 100b,
107b 100b display processing unit, 108b 100b display processing unit,
109a 100a recording control unit, 111 operation unit, 120 imaging optical system,
121: Variable-magnification optical system: 141: Imaging element: 142:
151 division information generation unit, 161 display devices, 162 display devices,
171 Recording medium

Claims (4)

An imaging unit that acquires captured image data at an arbitrary frame rate;
A division unit that divides captured image data;
First and second image processing apparatuses that perform image processing on divided image data divided by the dividing unit;
An image data transfer unit for transferring image data between the first and second image processing apparatuses; a display unit for displaying image data subjected to image processing;
Consists of
The first image processing device
A display memory for combining the first image data generated from the first divided image data and the second image data transferred by the data transfer means, and storing it as display data;
A display control unit which reads display data from the display memory and displays it on a display unit;
Equipped with
An imaging apparatus, wherein division positions of the division unit are set according to processing of a display control unit, and processing amounts of the first and second image processing apparatuses are determined.   The imaging apparatus according to claim 1, wherein the division position of the division unit is set according to an amount of display data read by the display control unit.   The imaging apparatus according to claim 1, wherein the dividing unit divides captured image data into upper and lower parts.   The imaging unit is characterized by sequentially reading out the captured images line by line, and controlling the processing amount of the image processing apparatus which performs processing of the late-captured captured image decreases as the display data amount increases. The imaging device according to claim 2, characterized in that