JP2019087950A - Imaging apparatus - Google Patents

Abstract

To allow for shortening of power supply reset time, while restraining electrical power consumption.SOLUTION: An imaging apparatus capable of decentralized image processing by multiple image processing circuits comprises imaging means, display means, recording means, a master image processing circuit, and single or multiple slave image processing circuits. Work memory of the master image processing circuit is a nonvolatile memory, the imaging means, the display means and the recording means are connected directly with the master image processing circuit, the master image processing circuit is responsible for simple development preprocessing for display, and recording processing, work memory of the slave image processing circuit is a volatile memory, the slave image processing circuit is responsible for development processing and compression processing of the recording image, and electric power supply can be blocked during a period where development processing and compression processing of the recording image do not occur.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an imaging device.

  In recent years, imaging devices capable of recording a large-scale image having a number of pixels have appeared. In the case of an imaging device with a small number of pixels, it has been common to process all images with a single system LSI. However, when dealing with an image with a large number of pixels, the image processing time will be longer in a single system LSI, so imaging devices are configured with a plurality of system LSIs according to the required frame rate, and image data is distributed and processed. There is a need. For example, Patent Document 1 proposes a technology for performing distributed processing of image data from a sensor with two processing circuits.

JP, 2013-219424, A

  As a work memory of a conventional system LSI of an imaging apparatus, a DRAM is generally adopted because it is superior in terms of access speed, memory capacity, and the like. Further, even when each of the plurality of system LSIs has its own dedicated work memory, each dedicated work memory is generally used by unifying only the DRAM without mixing other types of memories.

  However, since the DRAM is a volatile memory which loses internal data when the power is shut off, when the power of the entire system is turned off and then the power is turned on, all stored data in the DRAM is temporarily lost when the power is turned off. Then, since various data necessary for the system are stored again from the time of the next power-on, there is a problem that it takes time to restore the entire system.

  To address this issue, it is possible to use nonvolatile memory instead of volatile memory as a work memory, but in imaging devices, proprietary process information such as development process data remains in nonvolatile memory when the power is shut off. There is a problem of technical information leak.

  Also, when each dedicated work memory of a plurality of system LSIs is a DRAM, from the viewpoint of power saving, in a certain operation mode, the system LSIs sharing the used functions and the system LSIs sharing the unused functions are clearly divided. Configuration is required. By doing so, it is possible to suppress power consumption by controlling to turn on only the system LSI and DRAM of the in-use functional unit and turn off the power of the unused system LSI and DRAM. However, in a plurality of LSI systems, there is also considered a system in which all the functions do not operate when the power supply of part of the LSI is turned off. In this case, there is a problem that the power consumption can not be suppressed as described above.

  Therefore, an object of the present invention is to provide an imaging device capable of shortening the power supply recovery time, having no fear of technical information outflow, and suppressing power consumption.

  The imaging device, the display device, the recording device, the master image processing circuit, and one or more slave image processing circuits, the work memory of the master image processing circuit is a non-volatile memory, and the master image processing The imaging means, the display means, and the recording means are directly connected to the circuit, and the master image processing circuit is in charge of pre-development processing, simplified development processing for display, recording processing, and work memory of the slave image processing circuit. Is a volatile memory, and the slave image processing circuit is in charge of development processing and compression processing of a recorded image, and power of the slave image processing circuit can be shut off while development processing of the recorded image and compression processing are not generated.

  According to the present invention, the power supply recovery time can be shortened, there is no fear of the outflow of technical information, and power consumption can be suppressed.

BRIEF DESCRIPTION OF THE DRAWINGS The block diagram of the imaging device of 1st Example. FIG. 5 is a schematic view of an image data division method according to the first embodiment. The schematic diagram of the image data division method of 2nd Example. The block diagram of the imaging device of 2nd Example.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. In the first embodiment, a master image processing circuit and two slave image processing circuits are connected in a tree, one slave image processing circuit shares processing of a part of image data, and the remaining part of image data is the other. The example which a slave image processing circuit carries out processing assignment is shown. That is, in the case of processing a plurality of images such as still image continuous shooting and moving images continuously at high speed, the required frame rate can be realized.

  FIG. 1 is a block diagram of an imaging apparatus according to the first embodiment. First, an overview of the flow of image data will be described. The imaging data captured by the imaging sensor 101 is input to the first image processing circuit 200, and part of the image data to be described later among the image data subjected to arithmetic processing in the first image processing circuit is transmitted to the second image processing circuit 300 as the first. It is input via the serial bus 121. Similarly, part of the remaining image data of the image data subjected to arithmetic processing in the first image processing circuit 200 is input to the third image processing circuit 400 via the first serial bus 122.

  Then, the image data subjected to the arithmetic processing in the second image processing circuit is input to the first image processing circuit 200 through the second serial bus 126, and the image data subjected to the arithmetic processing in the third image processing circuit is the first It is input to the image processing circuit 200 via the second serial bus 127. Here, the first serial bus is assumed to be a standard such as LVDS, and the second serial bus is assumed to be a standard such as PCIe, but it is not limited to this. The image data subjected to arithmetic processing in each of the second and third image processing circuits is combined in the first image processing circuit, and then stored in the recording medium 103 via the recording interface 252.

  Here, a method of image data division from the first image processing circuit to the second and third image processing circuits will be described with reference to the schematic view of FIG. FIG. 2 shows a temporally continuous frame image sequence of moving image data. The image data does not have to be a moving image, and may be a plurality of high-speed continuous shot still images. Among the images of all the frames to be arithmetically processed by the first image processing circuit, data 850, 852, 854,... Of even numbered frames are transmitted to the second image processing circuit, and data 851, 853, 855 of odd numbered frames. Are sent to the third image processing circuit.

  Although it is assumed that the first, second and third image processing circuits have the same circuit configuration in this embodiment, they may have different circuit configurations. Each image processing circuit activates only necessary functions and performs sharing operation according to the role.

  The flow of image data of the imaging device of the first embodiment will be described in more detail. The imaging data captured by the imaging sensor 101 is input to the serial input interface 201 in the first image processing circuit 200. The image data output from the serial input interface is temporarily stored in the non-volatile memory 111 via the memory bus 271 and the memory interface 241.

  Next, the image data is read out from the non-volatile memory 111, and the development pre-processing unit 212 performs pixel defect interpolation, optical image distortion correction of a lens (not shown), and the like. Data necessary for pixel defect interpolation processing and optical image distortion correction processing in calculations such as pixel defect interpolation and optical image distortion correction in the pre-development processing unit are stored in advance in the non-volatile memory 111. The development preprocessing unit 212 executes pixel defect storage processing / optical image distortion correction processing while referring to the interpolation / correction data in the non-volatile memory 111. At this time, the data calculated by the development pre-processing unit is written to and read from the non-volatile memory through the memory bus 271 and the memory interface 241.

  The image data subjected to the pre-development processing is input to the simple developing unit 231 via the memory bus 271, and the simple development for live view display is performed. At this time, the arithmetic data of the simple developing unit is written to and read from the non-volatile memory via the memory bus 271 and the memory interface 241 as in the pre-development processing. Then, the data is converted to a data format for live view output by the display interface 232 and output to the display unit 102.

  Further, among the image data subjected to pre-development processing, data of the even-numbered frame shown in FIG. 2 is output to the first serial bus 121 through the serial output interface 268. Also in this case, the memory bus 271, the memory interface 241, and the non-volatile memory 111 are used as an image data relay unit, and only even-numbered frame data is selectively output from image data mixed with even-numbered frames and odd-numbered frames.

  Then, among the image data subjected to the pre-development processing, data of the odd-numbered frame shown in FIG. 2 is output to the first serial bus 122 through the serial output interface 268. Also in this case, the memory bus 271, the memory interface 241, and the non-volatile memory 111 are used as an image data relay unit, and only odd numbered frame data is selected and output from image data mixed with even numbered frames and odd numbered frames.

  Note that, in FIG. 1, the circuit blocks in the inactive state are indicated by hatching patterns, and in the first image processing circuit 200, it represents that the developing unit 221 and the compression unit 242 are in the inactive state. That is, in the case of the configuration of the imaging device of the first embodiment, the main functions of the first image processing circuit are pre-development processing, simple development, display processing, and recording processing. In the first image processing circuit 200, a developing unit 221 and a compressing unit 242 that share development processing of a recorded image are also disposed, but image data subjected to pre-development processing is not subjected to development processing and compression processing. , And through the serial output interface 268, to the first serial bus 121.

  So far, it is assumed that the function of the first image processing circuit is not very high load. Therefore, even a high frame rate at the time of moving image shooting or high-speed continuous shooting can be processed by one image processing circuit if the processing up to this point.

  Next, among the image data subjected to pre-development processing in the first image processing circuit, data of the even-numbered frame is input to the serial input interface 301 in the second image processing circuit 300. Here, in the second image processing circuit 300, the hatched pattern development pre-processing unit 312, simplified developing unit 331, display interface 332, recording interface 352, and serial output interface 368 are in a pause state. That is, in the case of the configuration of the imaging device of the first embodiment, the main functions of the second image processing circuit are development processing and compression processing of a recorded image.

  In the second image processing circuit 300, a pre-development processing unit 312 that shares pre-development processing is disposed, but input image data is input to the development unit 321 without passing through the pre-development processing unit 312. That is, the image data input to the serial input interface 301 is relayed to the memory bus 371, the memory interface 341, and the volatile memory 112, and the data is transferred to the developing unit 321. The image data input to the development unit 321 is subjected to development processing such as color balance adjustment and noise removal, and then input to the compression unit 342 and data compression is performed.

  In the developing process of the developing unit and the compression process of the compressing unit, the developing unit and the compressing unit execute the process while temporarily storing data during calculation in the volatile memory 112. At this time, the arithmetic data passes through the memory bus 371 and the memory interface 341 to write data to and read data from the volatile memory. Further, the developed and compressed image data is output to the second serial bus 126 through the high-speed serial input / output interface 364. Also in this case, the memory bus 371, the memory interface 341, and the non-volatile memory 112 are used as the image data relay unit, and the even-numbered frame image data is sequentially output.

  Next, among the image data subjected to pre-development processing in the first image processing circuit, data of the odd-numbered frame is input to the serial input interface 401 in the third image processing circuit 400. The details thereafter are the same as those of the second image processing circuit 300, and thus the description thereof is omitted.

  Up to this point, it is assumed that development and compression processing of a recorded image mainly carried by the second image processing circuit and the third image processing circuit is a high load. Therefore, one image processing circuit can not handle the high frame rate at the time of moving image shooting or high-speed continuous shooting. Therefore, image processing at a high frame rate is made possible by distributing the processing by the second image processing circuit and the third image processing circuit.

  The even-numbered frame image data subjected to the arithmetic processing in the second image processing circuit and the odd-numbered frame image data subjected to the arithmetic processing in the third image processing circuit pass through the second serial bus 126, 127 Input to the high-speed serial input / output interface 264. Then, the data is temporarily stored on the non-volatile memory 111 via the memory bus 271 and the memory interface 241. That is, even-numbered frame image data and odd-numbered frame image data are synthesized and stored on the non-volatile memory 111.

  Next, the composite image data in the non-volatile memory is subjected to data format conversion to a recording medium by the recording interface 252, and is stored in the recording medium 103. In the recording medium data format conversion, the recording interface performs processing while temporarily storing data in process in the non-volatile memory 111. At this time, processing data of the recording interface passes through the memory bus 271 and the memory interface 241, and data writing and reading are performed to the non-volatile memory.

  The above is a detailed description of the flow of image data of the imaging device of the first embodiment. Although the data division method from the first image processing circuit of this embodiment to the second and third image processing circuits is divided by the even and odd numbers of the frame numbers shown in the schematic view of FIG. 2, another method may be used. . For example, FIG. 3 shows a method of dividing one frame into two, an upper area 810 and a lower area 820. The division method is not limited to two, and may be three or more.

  As in the present embodiment, the image data of the first image processing circuit is divided into the second and third image processing circuits and processed, whereby the first image processing circuit and the second image processing circuit Between the first image processing circuit and the third image processing circuit. At the same time, the operation load of the second image processing circuit and the third image processing circuit can be reduced.

  In addition, as in the present embodiment, the working memory of the first image processing circuit is a non-volatile memory, and data necessary for pixel defect interpolation processing and optical image distortion correction processing are stored in advance, so that the recovery time from power off Can be shortened. If the memory is volatile memory, the information is lost when the power is turned off, so it is necessary to re-read data from another nonvolatile memory such as a flash ROM each time.

  Further, in the present embodiment, the sharing function of the second and third image processing circuits is only the development and compression of the image data. However, the second and third image processing circuits may be added with functions such as audio processing and shake correction processing. This is because development, image compression, sound processing, blur correction processing, and the like are data that are generally unrelated to the recovery time from the power-off.

  Further, by using the working memory of the second image processing circuit and the third image processing circuit as the volatile memory as in this embodiment, the image data during development and compression disappears when the power is turned off. Therefore, it is possible to prevent the flow of image processing technology unique to the manufacturer.

  Further, in the case of the configuration of this embodiment, when the frame rate of moving image or still image continuous shooting is low, the processing can be performed only with the first image processing circuit and the second image processing circuit, for example. Therefore, in the case as described above, the power supply of the third image processing circuit can be turned off, and power consumption can be suppressed.

  In the second embodiment, the master image processing circuit and two slave image processing circuits are connected in a daisy chain, part of the image data is shared by the first slave image processing circuit, and the remaining part of the image data is An example in which the second slave image processing circuit shares processing is shown. That is, in the case of processing a plurality of images such as still image continuous shooting and moving images continuously at high speed, the required frame rate can be realized. FIG. 3 is a block diagram of an imaging apparatus according to the second embodiment.

  First, an overview of the flow of image data will be described. The imaging data captured by the imaging sensor 101 is input to the first image processing circuit 200, and all the image data calculated and processed by the first image processing circuit are input to the second image processing circuit 300 via the first serial bus 121. . Then, in the second image processing circuit, development and compression of a part of the input image data are performed, and the data of the remaining portion of the input image data for which development and compression are not performed is sent to the third image processing circuit 400 Input via the serial bus 123 of FIG. Next, in the third image processing circuit, development and compression of the remaining portion in which development and compression of the input image data are not performed are performed in the second image processing circuit.

  Then, the third image processing circuit combines the image data subjected to the development and compression in the second image processing circuit and the image data subjected to the development and compression in the third image processing circuit, on the second serial bus 127. Through the first image processing circuit 200. Finally, in the first image processing circuit, the recording interface 252 performs pre-recording processing on all the input image data that has been processed for development and compression, and stores the processed data in the recording medium 103. Here, the first serial bus is assumed to be a standard such as LVDS, and the second serial bus is assumed to be a standard such as PCIe, but it is not limited to this.

  Here, the slave second image processing circuit and the division method of the input image to which each of the slave third image processing circuits applies development and compression are the same as in the first embodiment, so detailed description will be omitted. Here, the upper and lower divisions of FIG. 3 will be described as an example.

  Although it is assumed that the first, second and third image processing circuits have the same circuit configuration in this embodiment, they may have different circuit configurations. The first, second and third image processing circuits of the first embodiment and the first, second and third image processing circuits of the second embodiment all have the same circuit configuration here. Each image processing circuit activates only necessary functions and performs sharing operation according to the role.

  Next, the flow of image data of the imaging device of the second embodiment will be described in more detail. The operations of the imaging sensor 101 and the first image processing circuit 200 are the same as those in the first embodiment, and thus detailed description will be omitted.

  All image data subjected to pre-development processing in the first image processing circuit is input to the serial input interface 301 in the second image processing circuit 300. Here, in the second image processing circuit 300, the hatched pattern development pre-processing unit 312, simplified developing unit 331, display interface 332, recording interface 352, and serial output interface 368 are in a pause state. That is, in the case of the configuration of the imaging device of the second embodiment, the main functions of the second image processing circuit are development processing and compression processing of a recorded image.

  In the second image processing circuit 300, a pre-development processing unit 312 that shares pre-development processing is disposed, but input image data is input to the development unit 321 without passing through the pre-development processing unit 312. That is, first, all image data input to the serial input interface 301 is temporarily recorded in the volatile memory 112 via the memory bus 371 and the memory interface 341. Then, in the case of the configuration of the second embodiment, only the data in the upper region 810 shown in the schematic view of FIG. 3 is transferred from the volatile memory to the developing unit 321 via the memory interface and the memory bus. The image data input to the development unit 321 is subjected to development processing such as color balance adjustment and noise removal, and then input to the compression unit 342 and data compression is performed.

  In the developing process of the developing unit and the compression process of the compressing unit, the developing unit and the compressing unit execute the process while temporarily storing data during calculation in the volatile memory 112. At this time, the operation data is written to and read from the volatile memory through the memory bus 371 and the memory interface 341. Further, the data in the upper region 810 shown in the schematic view of FIG. 3 that has been developed and compressed is temporarily recorded in the volatile memory 112 via the memory bus 371 and the memory interface 341. Then, development in volatile memory, data in upper region 810 that has been compressed, data in development in volatile memory, data in lower region 820 that has not been compressed, are both performed via memory interface and memory bus, and serial output interface Output to 368.

  Next, both development and compressed data and development and non-compression data are input to the serial input interface 401 in the third image processing circuit 400 through the serial output interface 368 and the first serial bus 123. The details thereafter are the same as those of the second image processing circuit 300, and thus the description thereof is omitted. The difference is that in the third image processing circuit 400, development and compression processing is performed on the lower region 820 in which development and compression have not been performed in the second image processing circuit 300.

  Next, the image data processed by the second image processing circuit 300 and the third image processing circuit 400 passes through the high-speed serial input / output interface 464 and the second high-speed serial bus 127, and the high speed in the first image processing circuit 200. Input to serial I / O interface 264. The image data input to the first image processing circuit is temporarily stored on the non-volatile memory 111 via the memory bus 271 and the memory interface 241. Next, the composite image data in the non-volatile memory is subjected to data format conversion to a recording medium by the recording interface 252, and is stored in the recording medium 103. In the recording medium data format conversion, the recording interface performs processing while temporarily storing data in process in the non-volatile memory 111. At this time, the processing data of the recording interface passes through the memory bus 271 and the memory interface 241 to write data to and read data from the non-volatile memory.

  The above is the detailed description of the flow of image data of the imaging device of the second embodiment. The data division method in the second and third image processing circuits is not limited to the upper and lower two divisions shown in the schematic view of FIG. 3, but may be division by even and odd frames as shown in FIG.

  In the first embodiment, a tree connection configuration is shown, and in the second embodiment, a daisy chain connection configuration is shown. The daisy chain connection has a higher bus bandwidth between the image processing circuits than the tree connection, but has the advantage of simplifying the wiring between the image processing circuits and reducing the number of terminals of the master image processing circuit.

200 first image processing circuit 110 non-volatile memory 102 display unit

Claims (5)

Imaging means,
Display means,
Recording means,
A master image processing circuit and one or more slave image processing circuits,
The work memory of the master image processing circuit is a non-volatile memory,
The imaging means, the display means, and the recording means are directly connected to the master image processing circuit,
The master image processing circuit is responsible for pre-development processing, simple development processing for display, and recording processing,
The work memory of the slave image processing circuit is a volatile memory,
The slave image processing circuit is responsible for development processing and compression processing of a recorded image,
An image pickup apparatus characterized in that a power supply of a slave image processing circuit can be shut off during a period in which development processing and compression processing of a recorded image do not occur. There are a plurality of slave image processing circuits, all of the master image processing circuit and the slave image processing circuit are directly connected, and the master image processing circuit transmits divided image data to each slave image processing circuit. 2. The image pickup apparatus according to claim 1, wherein development processing and compression processing are distributed in each slave image processing circuit. There are a plurality of slave image processing circuits, all the image processing circuits are connected in a series connection starting from the master image processing circuit, and the master image processing circuit transmits all image data to the slave image processing circuit. Each slave image processing circuit performs development processing and compression processing only on a part of image data assigned to itself among image data, and passes image data not assigned to itself without processing. The image pickup apparatus according to claim 1, wherein distributed processing is performed by the slave image processing circuit.   The image pickup apparatus according to any one of claims 1 to 3, wherein the data to be recorded in the non-volatile memory is any one of a lens correction value, a sensor correction value, a monitor image, and medium recording data.   The image pickup apparatus according to any one of claims 1 to 3, wherein the data to be recorded in the volatile memory is any one of image operation data, sound operation data, and shake correction data.