JP2016054477A - Imaging device, control method and program thereof, and imaging element - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an imaging device capable of reducing load of processing image data, transmitted from an image sensor, at processing a region of a part of image data being imaged.SOLUTION: An imaging device includes imaging means for generating image data of a subject, and addition means for adding additional information, in which first region information for identifying a first pixel region of the image data and second region information for identifying a second pixel region contained in the first pixel region are included, to the image data. Furthermore, the imaging device includes transmission means for transmitting image data with added additional information, and image processing means for performing predetermined image processing by extracting the first pixel region and the second pixel region, identified by additional information obtained by receiving image data, from image data received through the transmission means.SELECTED DRAWING: Figure 3

Description

  The present invention relates to an imaging apparatus, a control method and program thereof, and an imaging element.

  As the image quality of an image sensor increases and the frame rate increases, an improvement in the transmission speed of an interface between the image sensor and a DSP (Digital Signal Processor) that processes image data captured by the image sensor is required. . For such a situation, Patent Document 1 discloses that image data is transmitted by generating a packet in which header information is added to pixel data on a plurality of transmission paths connecting between an imaging sensor and a DSP. Has been disclosed.

JP 2012-120158 A

  By the way, in a specific shooting mode such as live view enlarged display at the time of moving image shooting, a part of image data being shot may be enlarged and displayed in accordance with a user operation. In order to change the display area of the image being taken in this way, a part of the area (readout area) that changes as needed for recording and display is extracted from the image data transmitted from the image sensor and displayed in an enlarged manner. It is necessary to perform such processing. For this reason, the DSP needs to perform an operation for obtaining an area for performing enlarged display or the like as needed with respect to the obtained readout area. Therefore, it is assumed that the processing load of the DSP for processing a part of the read area increases as the image quality and the frame rate increase. In the technique disclosed in Patent Document 1, header information transmitted from the image sensor to the DSP is limited to control information such as the start / end of a frame and the line number to which pixel data belongs. Accordingly, the processing for all or a part of the reading area is not taken into consideration, and in order to perform the processing by the DSP, the control unit needs to separately control the DSP, and the processing is assumed to be complicated. Is done.

  The present invention has been made in view of the above-mentioned problems of the prior art. The present invention provides an imaging apparatus capable of reducing a processing burden on image data transmitted from an imaging sensor, a control method and program thereof, and imaging when processing is performed on a partial area of image data being captured. An object is to provide an element.

  In order to solve this problem, for example, an imaging apparatus of the present invention has the following configuration. That is, an imaging unit that generates image data of a subject, first area information that specifies a first pixel area of the image data, and a second area that specifies a second pixel area included in the first pixel area The image data is received from the addition means for adding the additional information including the area information to the image data, the transmission means for transmitting the image data to which the additional information is added, and the image data received through the transmission means. Image processing means for extracting a first pixel area and a second pixel area specified by the obtained additional information and performing predetermined image processing;

  According to the present invention, it is possible to reduce the processing load on the image data transmitted from the image sensor when processing is performed on a part of the image data being captured.

1 is a block diagram illustrating a functional configuration example of a digital camera 100 as an example of an imaging apparatus according to an embodiment of the present invention. The figure which shows typically the pixel area | region of the image sensor 102 which concerns on this embodiment. The figure which shows an example of the frame format of the transmission data which concerns on this embodiment Table showing contents and amount of header information according to this embodiment The figure which shows the relationship between the image area which concerns on this embodiment, and a read-out area | region (a), The figure which shows the relationship between a read-out area | region and a display area (b) A block diagram (a) showing a functional configuration example of the imaging sensor 102 and the signal processing unit 103 according to the first embodiment and a block diagram (b) showing a functional configuration example of the signal processing unit 103 according to the second embodiment. A flowchart showing a series of operations of transmission and display processing according to the present embodiment. FIG. 9 is a block diagram illustrating a functional configuration of an imaging apparatus 500 according to the third embodiment. FIG. 9 is a block diagram illustrating an example of a functional configuration of an AFE 501 according to the third embodiment. FIG. 9 is a block diagram illustrating a functional configuration of an imaging apparatus 600 according to the fourth embodiment. FIG. 9 is a block diagram illustrating an example of functional configurations of a DFE 601 and a signal processing unit 602 according to a fourth embodiment. The figure which shows an example of the laminated type image sensor which concerns on other embodiment. The block diagram which shows the function structural example of the mobile telephone as an example of the imaging device which concerns on other embodiment.

(Embodiment 1)
Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the drawings. In the following, an example in which the present invention is applied to an arbitrary digital camera capable of processing a partial area of an image being captured will be described as an example of an imaging apparatus. However, the present invention is not limited to a digital camera and can be applied to any device capable of performing the processing. These devices may include, for example, a mobile phone, a game machine, a tablet terminal, a personal computer, a clock-type or glasses-type information terminal.

(1 Configuration of digital camera 100)
FIG. 1 is a block diagram illustrating a functional configuration example of a digital camera 100 as an example of an imaging apparatus according to the present embodiment. One or more of the functional blocks shown in FIG. 1 may be realized by hardware such as an ASIC or a programmable logic array (PLA), or may be realized by a programmable processor such as a CPU or MPU executing software. May be. Further, it may be realized by a combination of software and hardware. Therefore, in the following description, even when different functional blocks are described as the operation subject, the same hardware can be realized as the subject.

  The photographing lens 101 is a photographing optical system that forms an image of light from a subject on the image sensor 102. The imaging sensor 102 includes an imaging element and has a configuration in which a plurality of pixels each having a photoelectric conversion element are two-dimensionally arranged. The subject image formed by the photographing lens 101 is photoelectrically converted by each pixel. Although there is no limitation in particular in the imaging sensor 102, CCD (Charge-Coupled Device) and CMOS (Complementary Metal Oxide Semiconductor) image sensor are used, for example. The imaging sensor 102 has an A / D conversion unit inside, and converts image signals in pixel units into digital signals to generate image data. As will be described later, the imaging sensor 102 converts the image data into packet data to which additional information such as header information is added, and transmits the packet data to the signal processing unit 103.

  The signal processing unit 103 includes a dedicated circuit such as a DSP, and performs various image processing such as development processing and gradation correction, compression / decompression processing, and the like on the image data input from the image sensor 102. When image data is input from the image sensor 102, the signal processing unit 103 refers to additional information added to the image data as will be described later, and extracts a predetermined region in the image data for enlarged display, for example. Process.

  The recording medium 104 includes a semiconductor memory, a magnetic disk, or the like that records captured image data.

  The display unit 105 displays captured images and various menu screens, and a liquid crystal display (LCD) or the like is used. The display unit 105 includes a touch panel and functions as an operation unit (not illustrated) that detects a user operation on the menu screen.

  The control unit 106 is, for example, a CPU or MPU, and controls the entire digital camera 100 by developing and executing a program stored in a ROM (not shown) in a work area of a memory (not shown). A RAM 107 is a storage medium that temporarily stores image data and the like, and is connected to the signal processing unit 103.

(2 Image data transmission method)
An example of a pixel region in the image sensor 102 will be described with reference to FIG. In the effective pixel area 201, effective pixels that photoelectrically convert light from a subject and output a signal corresponding to the amount of incident light are arranged. In addition to the effective pixels, the imaging sensor 102 includes an optical black pixel (OB pixel) that is shielded from light from the subject. The OB pixel receives a signal serving as a reference for correcting the output of the effective pixel. Output. The horizontal OB pixel area 202 and the vertical OB pixel area 203 are areas in which OB pixels are arranged. The horizontal OB pixel region 202 is a region in which the number of pixels in the vertical direction (direction along the column) is equal to the number of pixels in the vertical direction of the effective pixel region 201, for example, the horizontal direction of the effective pixel region 201 (along the row). (Direction) adjacent to the end (for example, the left side). The vertical OB pixel region 203 has the number of pixels in the horizontal direction equal to the sum of the number of pixels in the horizontal direction of the effective pixel region 201 and the horizontal OB pixel region 202, and the vertical end portion (for example, the upper side) of the effective pixel region 201. It is arranged in contact with. The effective pixel region 201, horizontal OB pixel region 202, and vertical OB pixel region 203 constitute a pixel region 200.

  Next, an example of a format used to transmit one frame of image data between the image sensor 102 and the signal processing unit 103 will be described with reference to FIG.

  Assuming that the arrangement of pixels in the horizontal direction (the direction along the row) is a line, the packet stores pixel data constituting one line including effective pixels and OB pixels. Accordingly, transmission of the entire image data of one frame is performed using a number of packets equal to or greater than the number of pixels in the vertical direction (direction along the column) of the effective pixel area.

  In the upper and middle stages of FIG. 3, a packet structure in which header information added when pixel data is transmitted line by line and data including the header information and pixel data for one line are made into one packet. Is shown. The correspondence between the pixel data output from the pixel area shown in FIG. 2 and the packet structure is schematically shown in the lower part of FIG.

  Header information 302 and footer information 304 are added before and after the pixel data 303. Further, a control code such as a start code 301 indicating the start of the packet is added before the header information 302, and an end code 305 indicating the end of the packet is added after the footer information 304. When the footer information 304 is not added, a control code such as the end code 305 is added after the pixel data. When the end code 305 is not added, the start code of the next frame is used as the end code.

  The header information 302 includes additional information related to pixel data of each line such as frame information 310, line information 311, skip position information 312, and display area information 313. FIG. 4 shows the content and information amount of each information included in the header information 302.

  The frame information 310 is information relating to the frame of the pixel data stored in the packet, and includes information indicating “frame start” or “frame end”. “Frame start” is 1-bit information indicating the head of the frame, and a value of 1 is set to “frame start” of a packet including pixel data of the first line of image data. In addition, a value of 0 is set in “frame start” of a packet including lines other than the first line. “Frame end” is 1-bit information indicating the end of the frame. The value “1” is set in the “frame end” of the packet including the pixel data of the terminal line of the effective pixel region, and the value “0” is set in the “frame end” of the packet including the pixel data of the line other than the terminal line. The

  The line information 311 is information related to the line of pixel data stored in the packet, and includes “line valid” and “line number”. “Line valid” is 1-bit information indicating whether or not the pixel data stored in the packet is pixel data of a line in the effective pixel area. A value of 1 is set in the “line valid” of the packet in which the pixel data of the line in the effective pixel area is stored, and 0 in the “line valid” of the packet in which the pixel data other than the line in the effective pixel area is stored. Value is set. The “line number” is 13-bit information indicating the line number of a line constituted by pixel data.

  The skip position information 312 is information relating to the skip position for reading out pixel data from the image data transmitted to the signal processing unit 103, and includes “vertical skip start”, “vertical skip end”, “horizontal skip start”, and “horizontal skip”. It consists of "Skip end". Each information is 13-bit information, “vertical skip start” is the start position of the skip position in the vertical direction (direction along the column), and “vertical skip end” is the vertical direction of the skip position (direction along the column) ) End position. Similarly, “horizontal skip start” represents the start position of the skip position in the horizontal direction (direction along the line), and “horizontal skip end” represents the end position of the skip position in the horizontal direction (direction along the line).

  The display area information 313 is information indicating pixel data included in the display area, and includes “vertical display start”, “vertical display end”, “horizontal display start”, and “horizontal display end”. Each information of the display area information is 13-bit information, “vertical display start” is the start position of the display area in the vertical direction (direction along the column), and “vertical display end” is the vertical direction of the display area. Indicates the end position (direction along the column). “Horizontal display start” indicates the start position of the display area in the horizontal direction (direction along the line), and “Horizontal display end” indicates the end position of the display area in the horizontal direction (direction along the line). Represent.

  The extension information 314 is composed of a “reservation” consisting of a 32-bit area provided for extension, and the data amount of the entire header information is 19 bytes.

  Further, a method for specifying the readout region and the display region based on the header information shown in FIGS. 3 and 4 from the pixel data corresponding to the pixel region shown in FIG. 2 will be described with reference to FIG. FIG. 5A shows a readout area specified by the pixel area 200 of the image sensor and the skip position information 312 of the header information 302.

  The read area 300 is an area extracted by the signal processing unit 103 for recording on the recording medium 104. The effective read area 204 in the read area 300 has four points indicated by the skip position information of the header information 302, that is, “vertical skip start”, “vertical skip end”, “horizontal skip start”, and “horizontal skip end”. It is specified as a region surrounded by the position of. The vertical OB read area 206 corresponding to the effective read area 204 is composed of a vertical OB read area 206a and a vertical OB read area 206b. The vertical OB readout area 206a is specified by the horizontal range indicated by the positions of “horizontal skip start” and “horizontal skip end” in the vertical OB pixel area. Further, the vertical OB readout area 206b is specified as an area arranged on the upper side of the horizontal OB pixel area, the number of pixels in the horizontal direction being equal to the number of pixels in the horizontal direction of the horizontal OB pixel area. The horizontal OB readout area 205 is specified by the vertical range indicated by the positions of “vertical skip start” and “vertical skip end” in the horizontal OB pixel area 202. In this way, the reading area 300 is extracted by specifying the effective reading area 204, the horizontal OB reading area 205, and the vertical OB reading area 206, respectively.

  Next, the relationship between the reading area 300 and the display area 400 will be described with reference to FIG. The display area 400 is an area used when a part of the readout area 300 is taken out and enlarged and displayed in a specific shooting mode such as live view enlarged display at the time of moving image shooting. The display area 400 is specified by the read area 300 and the display area information 313 of the header information 302. The display area 400 is determined as an area surrounded by four points indicated by “vertical display start”, “vertical display end”, “horizontal display start”, and “horizontal display end” of the display area information 313. .

(3 Functional configuration of image sensor 102 and transmission of image data)
A functional configuration example of the imaging sensor 102 and packet generation processing for transmitting image data by the transmission method described above will be described with reference to FIG.

  The imaging unit 102a photoelectrically converts the subject image incident on the pixel region 200 and outputs an analog signal in units of pixels. The A / D conversion unit 102b converts the obtained analog signal into a digital signal and outputs the digital signal, and the image data generation unit 102c generates the obtained digital signal as image data corresponding to the pixel region 200.

  The header generation unit 102d generates each piece of information such as the skip position information shown in FIG. 4 that is added to the pixel data for one line as header information based on the control information notified from the control unit 106. The control information notified from the control unit 106 includes, for example, control information related to the reading area and control information related to the display area. The control information related to the read area may be, for example, read area control information for an electronic image stabilization function. The control information related to the display area may include control information related to display magnification in live view enlarged display, for example. For example, the header generation unit 102d determines skip position information for specifying the reading area 300 based on the control information regarding the reading area for the electronic image stabilization function. Further, display area information for specifying the display area 400 is determined based on control information indicating a display magnification in live view enlarged display.

  The packet generation unit 102e generates a packet in which the header information generated by the header generation unit 102d is added to pixel data for one line. The output unit 102f outputs the packet generated by the packet generation unit 102e to the signal processing unit 103.

(4 Functional configuration of packet processing unit 103 and packet processing)
Next, a functional configuration example of the signal processing unit 103 and packet processing for receiving image data and performing signal processing by the transmission method described above will be described with reference to FIG.

  The input unit 103a receives a packet including pixel data for one line transmitted by the transmission method described above. The header extraction unit 103b extracts and analyzes header information from the packet received by the input unit 103a, and outputs header information and pixel data necessary for subsequent processing.

  The readout area extraction unit 103c extracts pixel data of the readout area based on the skip position information of the header information. Further, the readout area extraction unit 103c generates image data corresponding to the readout area from the extracted pixel data, and outputs the image data to the display area extraction unit 103d and the recording correction unit 103f.

  The display area extraction unit 103d extracts the display area 400 based on the display area information of the header information 302 out of the read areas extracted by the read area extraction unit 103c. As described above, by transmitting the image data to which the information for specifying the reading area and the display area is added from the imaging sensor 102, the signal processing unit 103 does not need to calculate the display area 400. Further, the control unit 106 does not need to notify the signal processing unit 103 of control information for controlling the reading area 300 or the display area 400. Therefore, in the signal processing unit 103, for example, the control on the display area 400 by the control unit 106 is unnecessary, and the control is simplified.

  The display correction unit 103e performs correction processing corresponding to the display characteristics of the display unit 105, for example, on the pixel data of the display region extracted by the display region extraction unit 103d, and outputs corrected image data.

  The recording correction unit 103f performs correction processing suitable for recording on the read area data extracted by the read area extraction unit 103c, and outputs corrected image data.

  As described above, by adding the information for specifying the reading area and the display area to the header information, the signal processing unit 103 can easily extract the display image and the recording image, and further the recording and display. Therefore, it is possible to easily apply the individual correction process.

(A series of operations related to image data transmission and display processing)
Next, a series of operations related to image data transmission and display processing will be described with reference to FIG. Note that, in the state where the digital camera 100 is performing live view display, for example, when a user operation on an operation unit (not shown) is detected, this processing is started.

  In step S <b> 701, the control unit 106 determines whether the user operation is an instruction to change the display area displayed on the display unit 105. The control unit 106 determines whether the user operation is an operation for changing the display area based on the operation information notified from the operation unit. When the display area is changed, the process proceeds to S702 and the display area is not changed. In the case, the process proceeds to S703.

  In step S <b> 702, the control unit 106 notifies the imaging sensor 102 of control information for changing the display area. The control information for changing the display area is, for example, control information related to the display magnification in live view enlarged display, and is updated to a display magnification corresponding to an instruction of a user operation, for example. Then, the control unit 106 notifies the image sensor 102 of the control information. Note that the control unit 106 separately determines a change in the readout area 300 for the electronic image stabilization function and the like, and separately notifies the imaging sensor 102 of the notification for the change.

  In step S <b> 703, the imaging sensor 102 performs imaging processing based on an instruction from the control unit 106. The image sensor 102 exposes light from the subject at appropriate time intervals to generate image data.

  In step S <b> 704, the imaging sensor 102 determines whether the display area 400 has changed with respect to the readout area 300. More specifically, the header generation unit 102d of the imaging sensor 102 specifies the display region 400 based on the control information for changing the display region acquired from the control unit 106. Since the header generation unit 102d also determines the readout area 300 based on the readout area control information separately notified from the control unit 106, the header generation unit 102d can specify the display area 400 corresponding to the readout area 300. If either the reading area 300 or the display area 400 has changed, the header generation unit 102d determines that the display area 400 for the reading area 300 has changed, and advances the process to S705. On the other hand, if it is determined that the area has not changed, the process proceeds to S706.

  In step S <b> 705, the imaging sensor 102 updates skip position information or display area information stored in the header information 302.

  In step S706, the imaging sensor 102 generates header information 302 and a packet including the header information. The header generation unit 102d generates header information 302 that stores skip position information or display area information. The header generation unit 102d stores display area information in which the display area information is the same as the read area 300 when the display area 400 and the read area 300 are the same because live view enlargement display or the like is not performed. Header information 302 is generated. Thereafter, the packet generation unit 102e generates a packet including the header information.

  In step S <b> 707, the imaging sensor 102 transmits the generated packet to the signal processing unit 103. The imaging sensor 102 repeats transmission of, for example, one line of packets at predetermined timing until an image of one frame is transmitted.

  In S708, the signal processing unit 103 sequentially receives the packets transmitted by the imaging sensor 102. In S709, the signal processing unit 103 extracts the pixel data of the readout region 300 from each received packet. More specifically, the readout region extraction unit 103c of the signal processing unit 103 extracts the pixel data of the readout region 300 based on the skip position information of the header information.

  In S710, the display area extraction unit 103d of the signal processing unit 103 extracts the pixel data of the display area 400 specified by the header information from the readout area 300.

  In step S711, the display correction unit 103e of the signal processing unit 103 generates a display image from the extracted pixel data of the display area and performs correction processing. Specifically, an interpolation process for adapting to the number of pixels of the display image and a correction process for the display image are performed based on the extracted pixel data.

  In step S <b> 712, when the display correction unit 103 e of the signal processing unit 103 completes the correction process for display, the display correction unit 103 e outputs a display image to the display unit 105. Then, when the signal processing unit 103 outputs the display image, the control unit 106 ends the series of processes.

  In the series of processes described above, the process of displaying the transmitted image data has been described. However, the process can also be applied to the process of recording the transmitted image data. That is, the recording correction unit 103f may perform a correction process for recording on the pixel data in the readout area extracted in S707 and output it to the recording medium 104.

  As described above, in this embodiment, when changing the display area 400 of the image data being shot, the imaging sensor 102 stores information indicating the readout area and the display area in the header information of the pixel data and transmits the information. I did it. Then, the signal processing unit 103 extracts the reading area and the display area based on the header information of the received pixel data. By doing so, it is possible to reduce the processing load on the image data transmitted from the image sensor when processing is performed on a part of the image data being captured. That is, the calculation for obtaining the display area 400 in the signal processing unit 103 becomes unnecessary. In addition, the control unit 106 notifies the imaging sensor 102 of control information for changing the display area. In this way, control of the signal processing unit 103 by the control unit 106 becomes unnecessary, and it becomes possible to simplify the display area control. Further, separate correction processing is applied to the read area 300 and the display area 400 extracted with reference to the header information. This makes it possible to easily apply different correction processes simply by referring to the header information. In other words, the correction parameter can be switched according to the area indicated by the header information.

(Embodiment 2)
Next, Embodiment 2 will be described. The digital camera 100 according to the second embodiment is different from the first embodiment in the configuration inside the signal processing unit 103. That is, the correction unit of the first embodiment includes correction units corresponding to the recording medium 104 and the display unit 105 shown in FIG. 6A, but in the second embodiment, as shown in FIG. Only a single correction unit 103g is provided. Since the configuration of the digital camera according to the second embodiment is the same except for the correction unit, the same components are denoted by the same reference numerals and redundant description is omitted, and differences will be mainly described.

  FIG. 6B is a block diagram illustrating a functional configuration of the signal processing unit 103 according to the second embodiment. When it is not necessary to switch the image processing between the display image data and the recording image data, the same image processing is performed by the correction unit 103g. In this case, even when the reading area and the display area are different, a single correction unit may perform image processing in a time division manner. In this way, as in the first embodiment, when processing is performed on a part of the image data being captured, it is possible to reduce the processing burden on the image data transmitted from the imaging sensor. . Furthermore, the correction unit can be shared and the circuit scale can be reduced.

(Embodiment 3)
Next, Embodiment 3 will be described. The digital camera 500 according to the third embodiment has the same basic configuration as that of the first embodiment, but is different from the first embodiment in that the digital camera 500 includes an AFE (Analog Front End) 501.

  FIG. 8 is a diagram illustrating an overall configuration of the digital camera 500, and FIG. 9 is a block diagram illustrating an example of functional configurations of the imaging sensor 502 and the AFE 501 in FIG. The imaging sensor 502 includes only the imaging unit 502a, and this functional block corresponds to the imaging unit 102a of the imaging sensor 102 in the first embodiment. The imaging sensor 502 outputs an analog signal in pixel units that has been subjected to photoelectric conversion.

  The AFE 501 includes an A / D conversion unit 501a, an image data generation unit 501b, a header generation unit 501c, a packet generation unit 501d, and an output unit 501e. Each configuration of reference numerals 501a to 501e corresponds to each configuration of reference numerals 102b to 102f described in the first embodiment. The AFE 501 is controlled by the control unit 106 and converts an analog signal in pixel units received from the image sensor 502 into a digital signal. Then, the same processing as that performed in each configuration of the reference numerals 102 b to 102 f in the first embodiment is performed, and the image data is transmitted to the signal processing unit 103. According to the present embodiment, as in the first embodiment, when processing is performed on a part of the image data being captured, it is possible to reduce the processing burden on the image data transmitted from the imaging sensor. . Furthermore, an image sensor that outputs an analog signal can be used by performing A / D conversion processing and header generation processing other than the image sensor. Further, by providing a functional block for image data transmission processing outside the imaging sensor, it is possible to apply the transmission processing and display control of the present invention to a simple imaging sensor.

(Embodiment 4)
Next, a fourth embodiment will be described. The digital camera 600 according to the fourth embodiment has the same basic configuration as that of the first embodiment, except that a digital front end (DFE) 601 is provided.

  FIG. 10 shows a functional configuration example of the digital camera 600. The packet transmitted from the imaging sensor 102 is processed using the DFE 601 and then subjected to correction processing by the signal processing unit 602. An example of the functional configuration of the DFE 601 is shown in FIG. The DFE 601 includes an input unit 601a, a header extraction unit 601b, a readout region extraction unit 601c, and a display region extraction unit 601d corresponding to the components 103a to 103d of the signal processing unit 103 described in the first embodiment. The display area data extracted by the display area extraction unit 601d is first subjected to correction processing such as gradation correction according to display characteristics by the first correction unit 601e. After that, the image is output from the DFE 601 and is subjected to image processing accompanying enlargement processing that matches the number of display pixels, for example, in the second correction unit 103h of the signal processing unit 602. The read area data extracted by the read area extraction unit 601c is first subjected to image processing such as gradation correction according to recording characteristics by the third correction unit 601f. Thereafter, the image is output from the DFE 601 and is subjected to predetermined image processing in accordance with, for example, the requirements of the recording medium 104 by the fourth correction unit 103 i of the signal processing unit 602. A part of the processing performed by the DFE 601 may be performed by the DSP.

  According to the present embodiment, as in the first embodiment, when processing is performed on a part of the image data being captured, it is possible to reduce the processing burden on the image data transmitted from the imaging sensor. . Furthermore, by adopting a configuration in which a plurality of image processes can be combined, a plurality of image processes for image data can be switched according to correction contents and data paths, and the efficiency of the image processing can be improved.

(Other embodiments)
The present invention can also be applied to other embodiments described below. For example, the present invention can be applied to a multilayer imaging sensor 1200 shown in FIG. As shown in FIG. 12, in the imaging sensor 1200 of this embodiment, a first semiconductor chip 1201 and a second semiconductor chip 1202 are stacked on a chip level. 12A shows an oblique projection, and FIG. 12B shows a top view of each chip. The first semiconductor chip 1201 includes, for example, a region including a pixel unit 1203 corresponding to the pixel region 200 of the imaging unit 102a in the first embodiment, and the second semiconductor chip 1202 for the high-speed logic process includes, for example, implementation. Portions 1204 and 1205 capable of high-speed processing including digital data such as a column A / D conversion circuit and a horizontal scanning circuit corresponding to the A / D conversion unit 102b in the first mode are included. In the configuration of FIG. 1 described above, for example, the imaging unit 102 a included in the imaging sensor 102 corresponds to the first semiconductor chip 1201. Also, the functional blocks of the A / D conversion unit 102b, the image data generation unit 102c, the header generation unit 102d, the packet generation unit 102e, and the output unit 102f included in the imaging sensor 102 are arranged on the second semiconductor chip 1202. What is necessary is just to comprise. The arrangement of the functional blocks on the first semiconductor chip 1201 and the second semiconductor chip is not limited to the above-described configuration. Further, the number of stacked semiconductor chips is not limited to two, and a configuration in which three or more semiconductor chips are stacked may be used.

  Furthermore, the present invention can also be applied to a mobile phone 1300 shown in FIG. 13 as an example of an imaging apparatus. FIG. 13 is a block diagram showing a functional configuration of the mobile phone 1300. The mobile phone 1300 has an electronic mail function, an Internet connection function, an image shooting / playback function, and the like in addition to a voice call function. The communication unit 1010 communicates audio data and image data with other telephones by a communication method according to a communication carrier contracted by the user. The voice processing unit 1020 converts voice data from the microphone 1030 into a format suitable for outgoing call and sends it to the communication unit 1010 during a voice call. Also, the voice processing unit 1020 decodes the voice data from the communication partner sent from the communication unit 1010 and sends it to the speaker 1040. The imaging unit 1050 captures an image of a subject and outputs pixel data. The imaging unit 1050 includes, for example, the imaging sensor 102 described above in the first embodiment. Here, the imaging sensor 102 may be the above-described stacked imaging sensor. Further, the imaging unit 1050 generates the above-described packet and transmits the packet to the image processing unit 1060. The image processing unit 1060 includes the signal processing unit 103 described above in the first embodiment, and processes the pixel data transmitted from the imaging unit 1050 at the time of capturing an image, and displays an image suitable for display and recording, respectively. 1070 and the memory IF 1110. In addition, when the recorded image is reproduced, the image processing unit 1060 processes the reproduced image and sends the processed image to the display unit 1070. The display unit 1070 includes a liquid crystal display panel of about several inches, and displays various screens according to instructions from the control unit 1090. The nonvolatile memory 1080 stores data such as address book information, e-mail data, and image data captured by the imaging unit 1050.

  The control unit 1090 includes a CPU, a memory, and the like, and controls each unit of the mobile phone 1300 according to a control program stored in a memory (not shown). The control unit 1090 includes the control unit 106 described above in each embodiment, and controls the imaging unit 1050 and the image processing unit 1060. The operation unit 1100 includes a power button, number keys, and other various operation keys for a user to input data. For example, a live view display operation can be performed. The memory IF 1110 records and reproduces various data on a recording medium 1120 such as a memory card. The external IF 1130 transmits data stored in the nonvolatile memory 1080 and the recording medium 1120 to the external device, and receives data transmitted from the external device. The external IF 1130 performs communication by a known communication method such as a wired communication method such as USB or wireless communication.

  Next, a voice call function in the mobile phone 1300 will be described. When making a call to the other party, the user operates the number key of the operation unit 1100 to input the number of the other party, or the address book stored in the nonvolatile memory 1080 is displayed on the display unit 1070, Select and instruct to make a call. When transmission is instructed, control unit 1090 transmits to communication party 1010 to communication unit 1010. When an incoming call is received, the communication unit 1010 outputs the other party's voice data to the voice processing unit 1020 and transmits the user's voice data to the other party.

  When transmitting an e-mail, the user uses the operation unit 1100 to instruct mail creation. When mail creation is instructed, control unit 1090 displays a screen for creating mail on display unit 1070. The user inputs a transmission destination address and text using the operation unit 1100 and instructs transmission. When the mail transmission is instructed, control unit 1090 sends address information and mail body data to communication unit 1010. The communication unit 1010 converts mail data into a format suitable for communication and sends it to a transmission destination. In addition, when receiving the electronic mail, the communication unit 1010 converts the received mail data into a format suitable for display and displays the data on the display unit 1070.

  Next, a photographing function in the mobile phone 1300 will be described. When the user operates the operation unit 1100 to set the shooting mode and then instructs to shoot a still image or a moving image, the imaging unit 1050 shoots the still image data or the moving image data and sends it to the image processing unit 1060. The image processing unit 1060 processes captured still image data and moving image data and stores them in the nonvolatile memory 1080. Further, the image processing unit 1060 sends the captured still image data and moving image data to the memory IF 1110. The memory IF 1110 stores still images and moving image data in the recording medium 1120.

  In addition, the mobile phone 1300 can transmit a file including still images and moving image data shot in this way as an attached file of an e-mail. Specifically, when an e-mail is transmitted, an image file stored in the nonvolatile memory 1080 or the recording medium 1120 is selected, and transmission is instructed as an attached file.

  In addition, the mobile phone 1300 can transmit a file including a captured still image or moving image data to an external device such as a PC or another telephone using the external IF 1130. The user operates the operation unit 1100 to select an image file stored in the nonvolatile memory 1080 or the recording medium 1120 and instruct transmission. The control unit 1090 controls the external IF 1130 to read the selected image file from the nonvolatile memory 1080 or the recording medium 1120 and transmit it to the external device.

  The present invention can also be realized by executing the following processing. That is, software (program) that realizes the functions of the above-described embodiments is supplied to a system or apparatus via a network or various storage media, and a computer (or CPU, MPU, or the like) of the system or apparatus reads the program. It is a process to be executed.

DESCRIPTION OF SYMBOLS 102 ... Imaging sensor, 103 ... Signal processing part, 104 ... Recording medium, 105 ... Display part, 106 ... Control part

Claims (15)

Imaging means for generating image data of a subject;
The additional information including the first area information specifying the first pixel area of the image data and the second area information specifying the second pixel area included in the first pixel area is the image information. Additional means for adding to the data;
Transmission means for transmitting the image data to which the additional information is added;
The first pixel area and the second pixel area specified by the additional information obtained by receiving the image data are extracted from the image data received via the transmission means, and a predetermined image is obtained. And an image processing means for performing processing. The image processing means further includes correction means for performing correction processing on the extracted first pixel area and the second pixel area,
The imaging apparatus according to claim 1, wherein the correction unit applies different correction processing to the first pixel region and the second pixel region. The adding means acquires control information for obtaining the first pixel region and the second pixel region,
The imaging apparatus according to claim 1, wherein the first area information and the second area information are added based on the acquired control information. An input unit for inputting an instruction to change the region to at least one of the first pixel region and the second pixel region;
4. The adding means acquires control information based on an instruction to change the area, and adds the first area information or the second area information based on the control information. The imaging device described in 1. The input means inputs a user operation serving as an instruction to change the region with respect to the second pixel region;
The imaging apparatus according to claim 4, wherein the adding unit acquires control information based on the user operation and adds the second region information based on the control information. The correction unit performs a correction process corresponding to the recording unit on the first pixel area, and performs a correction process corresponding to the display unit on the second pixel area,
The image processing means further comprises output means for outputting the first pixel area to the recording means and outputting the second pixel area to the display means. 2. The imaging device according to 2.   The imaging apparatus according to claim 6, wherein the correction unit performs a correction process corresponding to the recording unit and a correction process corresponding to the display unit using a single circuit.   The second area information is information indicating at least one of a start position in the direction along the column, an end position in the direction along the column, a start position in the direction along the row, and an end position in the direction along the row. The imaging device according to claim 1, wherein the imaging device is specified for the first pixel region.   The imaging apparatus according to claim 5, wherein the user operation includes an operation related to an enlarged display in a live view. Imaging means for generating image data of a subject;
The additional information including the first area information specifying the first pixel area of the image data and the second area information specifying the second pixel area included in the first pixel area is the image information. Additional means for adding to the data;
An image sensor comprising: transmission means for transmitting image data to which the additional information is added.   The adding means acquires control information for obtaining the first pixel area and the second pixel area, and based on the acquired control information, the first area information and the second area information are obtained. The imaging device according to claim 10, wherein the imaging device is added.   The second area information is information indicating at least one of a start position in the direction along the column, an end position in the direction along the column, a start position in the direction along the row, and an end position in the direction along the row. The imaging device according to claim 10, wherein the imaging element is specified for the first pixel region.   The image pickup device according to claim 10, wherein the image pickup device is a stacked image pickup device in which a plurality of semiconductor chips are stacked on each other. An imaging step in which the imaging means generates image data of the subject;
The additional information includes the first area information for specifying the first pixel area of the image data and the second area information for specifying the second pixel area included in the first pixel area. Is added to the image data;
A transmission step for transmitting image data to which the additional information is added; and
An image processing means extracts the first pixel area and the second pixel area specified by the additional information obtained by receiving the image data from the image data received via the transmission means. And an image processing step for performing predetermined image processing.   The program for functioning a computer as each means of the imaging device of any one of Claims 1 thru | or 9.

Images