JP6282136B2 - Imaging apparatus and control method thereof - Google Patents

Description

  The present invention relates to an imaging apparatus and a control method thereof, and more particularly to a technique for handling a moving image or a RAW image as a still image.

  In a conventional imaging device, a raw image, which is raw image information captured by an imaging sensor, is debayered (demosaiced) and converted into a signal composed of luminance and color difference, and noise removal and optical distortion correction are performed on each signal. Development processing such as image optimization is performed. The luminance signal and the color difference signal thus developed are compression encoded and stored in a storage medium.

  On the other hand, an imaging apparatus capable of storing a RAW image as data is known. RAW images require a large amount of data to be stored, but are preferred by advanced users because they have the advantage that correction and deterioration of the original image are minimized and can be edited after shooting. . As a configuration of an imaging apparatus capable of storing RAW image data, a technique has been proposed in which development parameters are stored together with RAW image data, and the RAW image is developed and reproduced using the development parameters during reproduction ( Patent Document 1).

JP 2011-244423 A

  In recent imaging apparatuses, with the evolution of imaging sensors, the number of pixels per image has increased significantly, and the number of images that can be taken continuously per second tends to increase. Therefore, the amount of information processing in development processing for RAW images has increased synergistically, and in order to perform real-time development processing in parallel with shooting, an arithmetic circuit with high information processing capability is required. Large power is required to drive the circuit.

  However, there is a problem in that the cost becomes high in a configuration in which an arithmetic circuit with high information processing capability can be driven with high output. Even when an arithmetic circuit with high information processing capability is used, there may occur a situation where high photographing performance cannot be executed due to the occupation of the circuit by development processing or the limitation of power consumption.

  On the other hand, as in the technique described in Patent Document 1, if the configuration is such that the RAW image is stored without being developed, the amount of information processing required for the developing process at the time of shooting can be reduced. It is thought that it becomes possible to perform higher shooting performance.

  However, when image data is stored in an undeveloped state like a RAW image, it is difficult to quickly reproduce and display the captured image. Further, in the conventional RAW image storage method, due to the special characteristics of the RAW image data, there is a problem that the stored RAW image cannot be reproduced (developed) by other devices, which is convenient for the user. It's not good.

  According to the present invention, in an imaging apparatus capable of storing RAW images, it is possible to realize high shooting performance while suppressing an increase in circuit size and an increase in power consumption required for information processing. ) Is intended to provide a technology that enables prompt display.

An imaging apparatus according to the present invention is an imaging apparatus that includes an imaging unit that generates a RAW image from a subject image, the first developing unit that develops the RAW image, and the RAW image that is the first developing unit. A second developing unit that develops the developing process with a higher load than the processing, and stores the RAW image, the image developed by the first developing unit, and the image developed by the second developing unit Storage means, extraction means for extracting subject information from the image developed by the first developing means, and a RAW image stored in the storage means based on the subject information by the second developing means. Table development priority determined, and control means wherein the developing the RAW image stored in the storage means to the developing descending order of priority in the second developing means, the image for developing And when displaying a reduced image of one frame or an image of one frame on the display means, an image developed by the first developing means is displayed, and a part of the image of one frame is displayed. And display processing means for displaying an image developed by the second developing means when enlarged and displayed .

  According to the present invention, development processing is simply performed at the time of shooting or the like when the information processing load increases, and the RAW image can be obtained with high accuracy in the interval between shootings when the information processing load is low or in an idle state. develop. As a result, it is possible to achieve high shooting performance while suppressing the increase in circuit size and power consumption required for information processing, and to quickly reproduce and display the shot video (image). It becomes possible.

1 is a block diagram illustrating a schematic configuration of an imaging apparatus according to an embodiment of the present invention. 10 is a flowchart of processing when the imaging apparatus is in a still image shooting mode. It is a figure which shows the example of a development priority setting file. It is a figure which shows the example of each file structure of a still image file and a RAW file of a still image. It is a flowchart of the process performed in the idle state of step S220 of FIG. 10 is a flowchart of processing when the imaging apparatus is in a still image reproduction mode. It is a figure which shows the example of a display process when an imaging device is in still image reproduction mode. It is a flowchart of a process when an imaging device is in video recording mode. It is a figure which shows the example of each file structure of a moving image file and a RAW file of a moving image. 10 is a flowchart of processing when the imaging apparatus is in a moving image reproduction mode.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. FIG. 1 is a block diagram illustrating a schematic configuration of an imaging apparatus 100 according to an embodiment of the present invention. The imaging apparatus 100 not only stores image information obtained by imaging a subject in a storage medium, but also reproduces the image information from the storage medium, develops and displays the image information, It has a function to transmit / receive to / from a server (cloud). Therefore, in other words, the imaging device 100 can be expressed as an image processing device, a storage device, a playback device, a storage playback device, or a communication device.

  The imaging device 100 includes a control unit 161, an operation unit 162, a display unit 123, an imaging optical unit 101, an imaging sensor 102, a sensor signal processing unit 103, a recognition unit 131, a camera control unit 104, and an evaluation value calculation unit 105. Further, the imaging apparatus 100 includes a developing unit 110, a RAW compression unit 113, a RAW expansion unit 114, and a buffer 115. Further, the imaging apparatus 100 includes a display processing unit 122, an external output terminal 124, a still image expansion unit 143, a moving image expansion unit 144, a still image compression unit 141, a moving image compression unit 142, a storage / playback unit 151, a storage medium 152, and a communication unit. 153 and a communication terminal 154.

  Although not shown, the control unit 161 includes a CPU, a ROM that stores a control program executed by the CPU, a RAM that functions as a work area for developing the control program and temporarily stores various data and parameters handled by the CPU. Etc. The control unit 161 controls the overall operation of the imaging apparatus 100 by executing a control program and controlling the operation and processing of each block constituting the imaging apparatus 100.

  The operation unit 162 includes input devices such as keys, buttons, and a touch panel used by the user to give instructions to the imaging apparatus 100. An operation signal generated by operating the operation unit 162 is detected by the control unit 161, and the control unit 161 controls the imaging apparatus 100 so that an operation corresponding to the input operation is executed. The display unit 123 includes a display device such as a liquid crystal display or an organic EL display, for example, a subject image being shot, a shot video (image), a menu screen for setting operating conditions of the imaging device 100, various information, and the like. Is displayed.

  The imaging optical unit 101 has various lenses such as a zoom lens and a focus lens (not shown). The imaging optical unit 101 may include a mechanical aperture, a shutter, and the like. The imaging sensor 102 is, for example, a CMOS sensor and includes a photoelectric conversion element, a Bayer array type color filter, and the like. When an operation unit 162 is instructed to start a shooting operation, an optical image of a subject to be imaged is input via the imaging optical unit 101 and formed on the imaging sensor 102. At this time, the camera control unit 104 controls the operations of the imaging optical unit 101 and the imaging sensor 102 at the time of shooting based on the evaluation value acquired from the evaluation value calculation unit 105 and the subject information extracted by the recognition unit 131. Details of the evaluation value calculated by the evaluation value calculation unit 105 and the subject information extracted by the recognition unit 131 will be described later.

  The sensor signal processing unit 103 performs pixel repair processing on the electrical signal photoelectrically converted by the imaging sensor 102. The pixel restoration process executed by the sensor signal processing unit 103 includes a process of interpolating a pixel to be restored using a peripheral pixel value with respect to a missing pixel or a low-reliability pixel value in the imaging sensor 102, or a predetermined process. Includes a process of subtracting the offset value. In the following description, a raw image (undeveloped image data) that is image information output from the sensor signal processing unit 103 is referred to as a “RAW image”. However, the definition of the RAW image is not limited to this, and even an image that has been subjected to a Debayer process, which will be described later, may be referred to as a RAW image until a nonlinear conversion process is performed. The RAW image output from the sensor signal processing unit 103 is supplied to the evaluation value calculation unit 105, the development unit 110, and the RAW compression unit 113.

  The RAW image is developed in the developing unit 110. The developing unit 110 includes a simple developing unit 111 (first developing unit), a high image quality developing unit 112 (second developing unit), and outputs from the developing unit 110 to the simple developing unit 111 and the high image quality developing unit 112. And a switch unit 121 for selecting from which to start. Both the simple development unit 111 and the high-quality development unit 112 perform debayer processing (demosaic processing) on the RAW image to convert the RAW image into a signal composed of luminance and color difference, and then remove noise included in each signal. Then, so-called development processing is performed such as correcting the optical distortion to optimize the image.

  The high image quality developing unit 112 performs each process with higher accuracy than the simple developing unit 111. Therefore, a developed image with higher image quality than the developed image after the developing process by the simple developing unit 111 is obtained by the developing process by the high image quality developing unit 112. However, on the other hand, the processing load for development processing is higher in the high-quality developing unit 112 than in the simple developing unit 111. Therefore, in the present embodiment, the high image quality developing unit 112 is not specialized for real-time development in parallel with shooting, and has a configuration capable of performing distributed processing over time after shooting. Therefore, the increase (peak) in circuit scale and power consumption is kept low.

  On the other hand, the simple developing unit 111 is configured so that the processing load for the developing process is smaller than that of the high-quality developing unit 112 so that the developing process can be performed at high speed. Therefore, although the image quality of the developed image generated by the simple developing unit 111 is lower than the image quality of the developed image generated by the high image quality developing unit 112, simple image development is used for developing a real-time RAW image in parallel with the shooting operation. The unit 111 is used.

  The switch unit 121 is switched by the control unit 161 in accordance with control according to the operation content instructed by the user through the operation unit 162 and the operation mode being executed. In the present embodiment, the simple developing unit 111 and the high image quality developing unit 112 exist independently in the developing unit 110, but simple development and high image quality can be achieved by switching the operation mode of one developing unit. It may be configured such that development can be performed exclusively. In the following description, the image information after simple development output from the simple development unit 111 is referred to as “simple image information”, and the image information after high quality development output from the high image quality development unit 112 is referred to as “high quality image”. It will be referred to as “information”. Further, in the following description, when the simple image information and the high-quality image information are not distinguished, “development image information” is appropriately used as a combined name.

  The developed image information output from the developing unit 110 is displayed on the display unit 123 after a predetermined display process is performed by the display processing unit 122. Further, the imaging apparatus 100 can output the developed image information to a display device (not shown) connected to the outside via the external output terminal 124. The external output terminal 124 is a general-purpose interface such as HDMI (registered trademark) or SDI.

  The developed image information (mainly simple image information) output from the developing unit 110 is supplied to the evaluation value calculating unit 105. The evaluation value calculation unit 105 calculates an evaluation value such as a focus state or an exposure state from a luminance value or a contrast value included in the RAW image acquired from the sensor signal processing unit 103 or the developed image information acquired from the developing unit 110. .

  The developed image information (mainly simple image information) output from the developing unit 110 is supplied to the recognition unit 131. The recognition unit 131 has a function of extracting and recognizing subject information from the developed image information. Specifically, it is detected whether or not there is a person's face in the image represented by the developed image information, and if there is a face, information indicating the position of the face is output, and further, face feature information ( For example, a specific person is authenticated (face authentication) based on the shape of the face, the positions of eyes, nose, mouth, and the like.

  The developed image information output from the developing unit 110 is supplied to the still image compressing unit 141 and the moving image compressing unit 142. The still image compression unit 141 is used when the developed image information is compressed as a still image, and the moving image compression unit 142 is used when the developed image information is compressed as a moving image. Each of the still image compression unit 141 and the moving image compression unit 142 performs high-efficiency encoding (compression encoding) on the developed image information to be compressed, generates image information whose information amount is compressed, and generates an image file (still image File, movie file). JPEG or the like is used for still image compression, and MPEG-2 or H.264 is used for moving image compression. H.264, H.C. 265 or the like can be used.

  The RAW compression unit 113 performs high-efficiency encoding on the RAW image output from the sensor signal processing unit 103 using a technique such as wavelet transform or differential encoding, and compresses the RAW file (hereinafter referred to as “compressed RAW file”). ) And stored in the buffer 115. The buffer 115 is a storage unit that temporarily stores the compressed RAW file. The compressed RAW file stored in the buffer 115 can be read later, but after the compressed RAW file is stored in the buffer 115, the compressed RAW file is moved to another storage medium to be stored, and at the same time, deleted from the buffer 115. It may be configured.

  The compressed RAW file, the still image file, and the moving image file are stored in the storage medium 152 by the storage / playback unit 151. The storage medium 152 is a built-in large-capacity memory, a hard disk, a removable memory card, or the like. The storage / playback unit 151 can also read a still image file, a moving image file, and a compressed RAW file from the storage medium 152. Further, the storage / playback unit 151 can write and read various files such as a still image file to an external storage or server (not shown) via the communication unit 153. The communication unit 153 has a function of accessing the Internet (not shown) or an external device by wireless communication or wired communication using the communication terminal 154.

  When a playback operation of a still image or a moving image is instructed through the operation unit 162, the storage / playback unit 151 acquires a predetermined file from the storage medium 152 or via the communication unit 153 under the control of the control unit 161. When the playback target is a RAW image, the storage / playback unit 151 stores the acquired compressed RAW file in the buffer 115. In addition, the storage / playback unit 151 supplies the acquired still image file to the still image expansion unit 143 when the playback target is a still image, and when the playback target is a moving image, the storage / playback unit 151 Supply to the extension unit 144. The RAW decompression unit 114 decodes and decompresses the acquired compressed RAW file to generate a decompressed RAW file. The RAW expansion unit 114 supplies the generated expanded RAW file to the simple development unit 111 or the high image quality development unit 112 of the development unit 110. The still image decompressing unit 143 decodes and decompresses the acquired still image file, and supplies it to the display processing unit 122 as a playback image of the still image. The moving image extension unit 144 decodes and expands the acquired moving image file, and supplies it to the display processing unit 122 as a reproduction image of the moving image. The display processing unit 122 performs necessary display processing on the acquired still image or moving image playback image and displays the display on the display unit 123.

  The imaging apparatus 100 configured as described above has four operation modes: a still image shooting mode, a still image playback mode, a moving image shooting mode, and a moving image playback mode. Each of these operation modes can be switched via the idle state, and can be switched by the operation and instruction of the operation unit 162 by the user. Each of these operation modes may be automatically switched according to the determination of the control unit 161.

  FIG. 2 is a flowchart of processing when the imaging apparatus 100 is in the still image shooting mode. Each processing shown in FIG. 2 is performed by causing the CPU of the control unit 161 to develop a program stored in the ROM into the RAM and executing the program, so that each block of the imaging apparatus 100 performs a predetermined operation under the control of the control unit 161. This is realized by executing the process.

  When processing in the still image shooting mode is started, in step S201, the control unit 161 determines whether or not the information processing load (hereinafter referred to as “processing load”) in the imaging apparatus 100 is small. When the processing load is small (YES in S201), the control unit 161 determines that the imaging apparatus 100 is in the idle state, proceeds to step S220 to execute the processing in the idle state, and the processing load is not small. (NO in S201), the process proceeds to step S202. For example, since the processing load is high during high-speed continuous shooting of still images, the processing always proceeds to step S202. On the other hand, when performing normal single shooting, for example, half of the interval between the first shooting and the second shooting (between the shooting of one still image and the shooting of the next still image) At a frequency, the process proceeds to step S220.

  In step S202, the camera control unit 104 controls the operations of the imaging optical unit 101 and the imaging sensor 102 so that shooting is performed under suitable conditions (camera control). Specifically, the lens of the imaging optical unit 101 is moved in accordance with the user's zoom and focus instructions, and the readout area of the imaging sensor 102 is set in accordance with the instruction for the number of pixels to be captured. Further, based on the evaluation value calculated by the evaluation value calculation unit 105 and the subject information detected by the recognition unit 131, control such as focus adjustment and tracking for a specific subject is performed.

  In the subsequent step S203, the sensor signal processing unit 103 uses signal processing for pixel restoration (peripheral pixel values for missing pixel values and low-reliability pixel values) for the electrical signals output from the image sensor 102. Interpolation processing, predetermined offset value subtraction processing, etc.). As a result, a RAW image is output from the sensor signal processing unit 103. In subsequent step S204, the simple developing unit 111 develops the RAW image. At this time, the control unit 161 sets the switch unit 121 so that the simple image information generated by the simple development unit 111 is output from the development unit 110.

  Specifically, the simple development unit 111 specifically limits the image size after development to 2 million pixels or less, and reduces or eliminates noise removal and optical distortion correction to a limited process. Realizes high-speed processing and simple processing. As a result, the imaging apparatus 100 can achieve shooting with a performance of, for example, 60 frames per second of 2 million pixels with a small circuit scale and low power consumption.

  The simple image information output from the simple developing unit 111 is supplied to the evaluation value calculating unit 105. Thereby, in step S205, the evaluation value calculation unit 105 calculates evaluation values such as a focus state and an exposure state from the simple image information. Note that the evaluation value calculation unit 105 may acquire a RAW image from the sensor signal processing unit 103 and calculate the evaluation value from the RAW image. Further, the simple image information output from the simple development unit 111 is supplied to the recognition unit 131. Thereby, in step S206, the recognition unit 131 performs subject detection based on the simple image information, generates subject information, and outputs the subject information. Further, the simple image information output from the simple developing unit 111 is supplied to the display processing unit 122. Accordingly, in step S207, the display processing unit 122 generates a display image from the acquired simple image information, and outputs the display image to another display device such as an external television via the display unit 123 or the external output terminal 124. In this way, image display is performed.

  The display image by the display unit 123 is used for live view display (shooting through image display) for the user to appropriately frame the subject in the still image shooting mode. Further, the display processing unit 122 utilizes the evaluation value supplied from the evaluation value calculation unit 105 and the subject information supplied from the recognition unit 131, for example, marking display or recognition on a focused area on the display image. A frame can be displayed at the position of the face.

  In subsequent step S208, the control unit 161 determines whether or not a shooting instruction is input from the user. The shooting instruction is performed by pressing a shutter button (not shown) or touching the touch panel when the display unit 123 has a touch panel function. If there is no shooting instruction (NO in S208), control unit 161 returns the process to step S201, and if there is a shooting instruction (YES in S208), the process proceeds to step S209.

  In response to the shooting instruction in step S208, the simplified image information developed by the simplified development unit 111 is supplied to the still image compression unit 141. Accordingly, in step S209, the still image compression unit 141 performs high-efficiency encoding processing (still image compression) on the acquired simple image information using a known still image compression technique such as JPEG, and the still image file Is generated. In subsequent step S <b> 210, the storage / reproduction unit 151 stores the still image file generated in step S <b> 309 in the storage medium 152.

  In response to the shooting instruction in step S <b> 208, the RAW compression unit 113 acquires a RAW image corresponding to the shot still image from the sensor signal processing unit 103. Therefore, in step S211, the RAW compression unit 113 performs high-efficiency encoding processing (RAW compression) on the acquired RAW image, generates a RAW file, and stores the RAW file in the buffer 115.

  Note that the high-efficiency encoding process performed by the RAW compression unit 113 may be lossy encoding or lossless encoding. Alternatively, the RAW compression by the RAW compression unit 113 may be omitted, and the RAW image may be output through without being compressed. In this embodiment, regardless of the presence or absence of RAW compression, a RAW file that can be restored as a high-quality file that does not significantly impair the information of the RAW image output from the sensor signal processing unit 103 is generated.

  In step S212, the storage / playback unit 151 stores the RAW file generated in step S211 in the storage medium 152. In steps S210 and S212, the storage / playback unit 151 may transmit and store the still image file and / or the RAW file to the external storage through the communication unit 153 and the communication terminal 154.

  In subsequent step S213, the control unit 161 generates and updates a development priority setting file for managing development priority. The development priority is a parameter indicating which image is to be developed first when there is an image that has not been developed by the plurality of high-quality development units 112 in the follow-up development process described later. In the present embodiment, the development priority is set according to the number of faces recognized by the recognition unit 131 and the result of person authentication. Specifically, the development priority is set higher as the number of faces increases, and the development priority becomes higher as the number of people whose faces are registered in advance (the more people who have succeeded in face authentication) increases. Is set. For example, 1 point is added for each number of recognized faces, and 2 points are added if the person is a registered person, and the total point calculated for each image is set as the development priority. As a result, the development priority is higher for portraits than for landscape photos, and is higher for photos that show a registered person (for example, family or own photos). .

  A list in which the development priority is associated with the RAW file is created as a development priority file and stored in the storage medium 152. If a development priority file already exists, the development priority file is updated by adding the created development priority file to the end thereof.

  FIG. 3 is a diagram illustrating an example of a development priority setting file. As shown in FIG. 3, the development priority setting file includes data in which the file name of the RAW file and the development priority are set. When the development priority setting file is generated and updated in step S213, the control unit 161 returns the process to step S201.

  Here, the file structure of each of the still image file and the still image RAW file will be described. 4A is a diagram illustrating an example of a file structure of a still image file, and FIG. 4B is a diagram illustrating an example of a file structure of a RAW file of a still image.

  As shown in FIG. 4A, the still image file 400 includes a header portion 401, a metadata portion 402, and a compressed data portion 403. The header portion 401 includes an identification code indicating that this file is a still image file format. The metadata unit 402 includes a RAW file name 404, a development status 405, and shooting metadata 406. A RAW file name 404 is a file name of a RAW file of a still image generated at the same time as the still image file. The development status 405 is information indicating that the still image file 400 has been developed by the simple development unit 111 or the high image quality development unit 112. As an example, the development status 405 is set with a flag for identifying whether the still image file is generated by the simple developing unit 111 or the high-quality developing unit 112. The shooting metadata 406 includes evaluation values calculated by the evaluation value calculation unit 105, subject information detected by the recognition unit 131, information at the time of shooting (lens type identification information of the imaging optical unit 101 and sensor type identification information of the imaging sensor 102). Etc.). Although not shown, the metadata unit 402 further displays an identification code of the storage medium 152 storing the RAW file generated simultaneously with the still image file 400, path information of the stored folder, and the like. May be included. The compressed data section 403 includes still image compressed data that has been encoded with high efficiency.

  A still image RAW file 410 shown in FIG. 4B includes a header portion 411, a metadata portion 412, and a compressed data portion 413. The header portion 411 includes an identification code indicating that this file is in a RAW file format. The metadata unit 412 includes a file name 414, a development status 415, and shooting metadata 416. A file name 414 is a file name of a still image file generated simultaneously with the RAW file. The development status 415 is information indicating that the still image file indicated by the file name 414 has been developed by the simple development unit 111 or the high image quality development unit 112. The shooting metadata 416 includes evaluation values calculated by the evaluation value calculation unit 105, subject information detected by the recognition unit 131, and information at the time of shooting (lens type identification information of the imaging optical unit 101 and sensor type identification information of the imaging sensor 102). Etc.). Although not shown, the metadata unit 412 includes an identification code of the storage medium 152 in which the still image file generated simultaneously with the still image RAW file 410 is stored, path information of the stored folder, and the like. May further be included. Further, the metadata unit 412 may store data obtained by converting the still image file 400 generated simultaneously with the RAW file 410 into metadata. The compressed data unit 413 includes compressed data obtained by performing high-efficiency encoding of a RAW image, but may be RAW image data that is not compressed.

  Note that the file structure shown in FIG. 4 is an example, and a configuration conforming to a standard such as DCF or EXIF may be used.

  As described above, in the still image shooting mode, the imaging apparatus 100 performs the development processing for the still image file generated in response to the live view display or the shooting instruction until the shooting instruction is given, by the simple developing unit 111. In addition, the imaging apparatus 100 generates a RAW file in response to a still image shooting instruction, but development processing is not required for generating the RAW file. Therefore, the imaging apparatus 100 can store a RAW file with low power consumption by using a small circuit while increasing the number of still images and the continuous shooting speed.

  Next, step S220 in FIG. 2 will be described. FIG. 5 is a flowchart of the process executed in the idle state in step S220. Each process shown in FIG. 5 is performed by causing the CPU of the control unit 161 to develop and execute a program stored in the ROM on the RAM, so that each block of the imaging apparatus 100 performs a predetermined operation under the control of the control unit 161. This is realized by executing the process.

  When the imaging apparatus 100 is in the idle state, in step S501, the control unit 161 determines whether to perform follow-up development according to the user setting. If the follow-up development is not performed (NO in S501), the control unit 161 advances the process to step S502. If the follow-up development is performed (YES in S501), the control unit 161 advances the process to step S520.

  In steps S502, S503, S504, and S505, the control unit 161 determines which operation mode the user has selected from the four operation modes of the still image shooting mode, the still image playback mode, the movie shooting mode, and the movie playback mode. . Then, the control unit 161 proceeds the process to one of steps S510, S511, S512, and S513 in order to operate the imaging apparatus 100 in the selected operation mode. The processing of steps S502 to S505 is returned to step S501 when the result of all the determinations is “NO” regardless of the order.

  The processes in steps S520 to S527 are processes related to the follow-up development. Before describing these processes, the follow-up development will be described. In the present embodiment, “chase development” refers to a high-quality still image that has been subjected to development processing again using a RAW file stored in the buffer 115 or the storage medium 152 as a source after the photographing operation is completed. This refers to the process of generating files and display images. The follow-up development targets both RAW files of still images and moving images, but in the following description, a RAW file of still images is taken as an example.

  As already described, since the still image file generated at the time of shooting is developed by the simple development unit 111, the image quality is not high, and thus it is effective as a rough confirmation of the shooting contents. It may not be sufficient for checking image details or for printouts. On the other hand, the RAW file that is generated at the same time as the still image file does not greatly impair the image information, but it is data before development processing, so it cannot respond immediately to image display or printout, and development processing is not possible. It takes time to do it. In addition, since the RAW file is not a widely spread file such as JPEG, the playback environment in which the RAW file can be handled is also limited.

  Therefore, in this embodiment, the chasing development is performed by reading a stored RAW file, performing a high-quality development process by the high-quality development unit 112, and then performing a predetermined compression process or the like to generate a still image file generated. It is executed by storing it in the storage medium 152 or the like. Then, such chasing development is executed in a waiting state for user operation such as between shooting and in an idle state (sleep state), that is, in a state where the processing load of the imaging device 100 is relatively small. This eliminates the need for development processing each time a high-quality image playback request, such as a detailed confirmation display or printout later, is possible in the same general usage environment as conventional still image files. Utilization becomes possible. The follow-up development is not limited to being manually performed, but the configuration in which the control unit 161 automatically executes increases the convenience for the user.

  Returning to the description of the processing in steps S520 to S527. As described with reference to FIG. 4, the storage medium 152 and the like store a set of a still image file and a RAW file for one shooting instruction. Therefore, in step S520, the control unit 161 determines whether there is a RAW file that has not been subjected to development processing (follow-up development) in the high-quality image development unit 112. The determination method includes the following method. In the present embodiment, as will be described later, a high-quality still image file generated by chasing and developing a RAW file is stored in the storage medium 152 or the like in a form of overwriting the still image file generated by the simple development unit 111. Remembered. Therefore, the development status 405 stored in the metadata section 402 of the still image file 400 indicates whether the still image file has been generated by the simple development section 111 or has been generated by the high image quality development section 112 by follow-up development. Set a flag to identify Thereby, the control part 161 can perform determination of step S520 with reference to the flag. Further, a flag indicating whether or not the follow-up development is completed is set in the development status 415 of the still image RAW file 410, and the control unit 161 may refer to the flag and perform the determination in step S520. . Alternatively, a separate table file indicating the development processing state may be prepared for a series of still images taken, and the control unit 161 may refer to this table file and perform the determination in step S520.

  If there is an unprocessed RAW file (YES in S520), the control unit 161 advances the process to Step S521. If there is no unprocessed RAW file (NO in S520), the control unit 161 returns the process to Step S502. In step S <b> 521, the control unit 161 selects a RAW file for a still image to be chased and developed. The RAW file is selected by referring to the development priority setting file shown in FIG. 3 and selecting the RAW file having the highest development priority. In the development priority setting file, data relating to the selected RAW file is deleted because it is no longer necessary, and the development priority setting file is updated accordingly.

  In subsequent step S522, control unit 161 determines whether or not the RAW file corresponding to the image selected in step S521 is buffered in buffer 115. If the RAW file is buffered (YES in S522), the control unit 161 advances the process to step S524. If the RAW file is not buffered (NO in S522), the control unit 161 advances the process to step S523.

  In step S523, the control unit 161 reads from the storage medium 152 or the like the RAW file that is the target of the follow-up development. In the buffer 115, the data is updated so that the new image captured in the still image capturing mode is preferentially retained, and the images captured in the past are deleted from the buffer 115 in order. Thereby, since the image captured immediately before is always held in the buffer 115, step S523 can often be skipped, and the processing speed can be increased. Also, by performing the follow-up development from the image taken immediately before, the image held in the buffer 115 can be preferentially processed, and the processing efficiency can be improved. .

  In step S524, the RAW decompression unit 114 decompresses the RAW file read from the buffer 115 or the storage medium 152, and restores the RAW image. In subsequent step S525, the high image quality developing unit 112 develops the restored RAW image with high image quality. The size (number of pixels) of the image developed by the high-quality developing unit 112 is the entire size read from the image sensor 102 or the size set by the user, and is much larger than the image developed by the simple developing unit 111. High quality (high image quality).

  Since the high image quality development unit 112 performs development processing with higher accuracy than the simple development unit 111, a high quality image can be obtained. On the other hand, in the high image quality developing unit 112, the processing load is larger than that in the simple developing unit 111. In the present embodiment, the development processing in the high-quality development unit 112 is performed in real time in parallel with shooting, and is performed when time is available, thereby increasing the circuit scale and power consumption. It is possible to suppress the increase of

  The high quality image information generated by the high quality development unit 112 is output to the display processing unit 122 and the still image compression unit 141 via the switch unit 121. Therefore, in step S526, the still image compression unit 141 performs high-efficiency encoding processing (still image compression) on the acquired high-quality image information to generate a high-quality still image file.

  In subsequent step S527, the storage / playback unit 151 stores the high-quality still image file output from the still image compression unit 141 in the storage medium 152 or the like. At that time, the storage / playback unit 151 assigns the same file name to the still image file by simple development stored at the same time as the original RAW file to the high-quality still image file, and the still image file by simple development. Is overwritten. Therefore, the still image file by simple development is deleted (erased). Then, the storage / reproduction unit 151 indicates that the development status 415 in the metadata unit 412 in the RAW file that is the source of the follow-up development has been developed in the high-quality development unit 112 (follow-up development). Change to information (flag) and update the RAW file. Note that the high-quality still image file stored in the storage medium 152 in step S527 has the same file structure as the still image file 400 shown in FIG.

  After step S527, the control unit 161 returns the process to step S502. As a result, if there is a RAW file that has not been subjected to the follow-up development, the development process by the high-quality development unit 112 is continued.

  As described above, in the present embodiment, the imaging apparatus 100 is chased when the processing load is relatively small, such as waiting for an operation by the user in an idle state or the like when in the shooting mode or in the playback mode. Development (development processing by the high image quality development unit 112) is executed. Then, the still image file by the simple development at the time of shooting is replaced with the still image file by the high quality development based on the RAW file. In the above description, the processing for still images is taken up, but the processing for moving images is performed in the same manner. Therefore, a moving image file by simple development at the time of shooting is also replaced with a moving image file by high image quality development using a RAW file. This eliminates the need to perform development processing each time a high-quality playback such as a detailed confirmation display or printout is required, and provides the same operational feeling and use as when using a still image file. A feeling can be obtained.

  If the number of shots (shooting frequency) is larger than the number of shots to be processed (processing capability) in follow-up development, there are a plurality of undeveloped images. However, as described above, the development priority is set and development is performed. Follow-up development is performed in the order according to the priority. Such a configuration is particularly useful when, for example, a still image file is shared with an external device via the communication unit 153. In other words, in the present embodiment, the development priority is higher for a person photo than for a landscape photo, and is higher when a plurality of people are shown in the person photo and a registered person is shown. People are often family and friends. Therefore, when taking a group photo with a family member or friend immediately after taking it and enjoying it while watching the photo together, the group photo is preferentially developed with higher image quality than other photos. Therefore, it is possible to quickly share an image. Thereby, there is an advantage that the waiting time for the development processing can be minimized for the other party sharing the image.

  Next, processing when the imaging apparatus 100 is in the still image playback mode will be described. FIG. 6 is a flowchart of processing when the imaging apparatus 100 is in the still image playback mode. Each process illustrated in FIG. 6 is performed by causing the CPU of the control unit 161 to expand a program stored in the ROM to the RAM and executing the program so that each block of the imaging apparatus 100 performs a predetermined operation under the control of the control unit 161. This is realized by executing the process.

  When the still image reproduction mode process is started, first, in step S601, the control unit 161 determines whether or not the processing load on the imaging apparatus 100 is small. When the processing load is small (YES in S601), the control unit 161 determines that the imaging apparatus 100 is in the idle state, proceeds to step S610 to execute the processing in the idle state, and the processing load is not small. (NO in S601), the process proceeds to step S602. For example, since the processing load is small while waiting for a user operation such as a playback instruction, the process proceeds to step S610, and when still image playback is started by a user operation (including a state during playback). Since the processing load is large, the process proceeds to step S602. In step S610, the process described above with reference to the flowchart of FIG. 5 is executed.

  In step S602, the control unit 161 determines whether an enlargement display instruction has been received from the user for the image to be reproduced. If there is an enlarged display instruction (YES in S602), control unit 161 advances the process to step S603, and if there is no enlarged display instruction (NO in S602), advances the process to step S620.

  Here, the display form of the enlarged display will be described. FIG. 7 is a diagram illustrating an example of display processing in the still image reproduction mode. Each of the first display screen 700, the second display screen 710, and the third display screen 720 shown in each drawing of FIG. 7 is a screen such as a liquid crystal display or an organic EL display included in the display unit 123.

  On the first display screen 700 in FIG. 7A, six reduced images 701 are displayed. On the second display screen 710 in FIG. 7B, a standard image 711 in which one of the six reduced images 701 shown in FIG. 7A is displayed on the entire screen is displayed. In the third display screen 720 of FIG. 7C, a partial area of one of the six reduced images 701 shown in FIG. 7A (here, the image shown in FIG. 7B). An enlarged image 721 in which is enlarged is displayed. For example, when confirming whether or not the focus is appropriate immediately after shooting, a method of enlarging and displaying the details of the subject image as in the third display screen 720 is generally performed.

  When the enlarged image 721 is displayed as in the third display screen 720, the process proceeds from step S602 to step S603. On the other hand, when displaying one or a plurality of reduced images 701 obtained by reducing an image of one frame as in the first display screen 700, the process proceeds from step S602 to step S620. When the standard image 711 corresponding to an image of one frame is displayed as in the second display screen 710, the number of display pixels is the number of pixels of the still image file by simple development (in the above example, 2 million If it is equal to or less than (pixel or less), the process proceeds from step S602 to step S620. This is because, when the display image is sufficient for the number of pixels of the still image file by simple development, even if the still image file is developed by the simple development unit 111, it can be displayed with sufficient image quality. Because.

  In step S620, the storage / playback unit 151 reads the still image file to be played back from the storage medium 152 or the like, and in the subsequent step S621, the still image decompression unit 143 decodes and decompresses the still image file. Thereafter, the process proceeds to step S608. In step S608, the display processing unit 122 displays an image as shown in FIG. 7A or 7B.

  When performing enlarged display, the displayed image may not be enough for the number of pixels of the still image file by simple development. In other words, display using a still image file by simple development causes a decrease in resolution. Therefore, in step S603 in the case of performing enlarged display, the control unit 161 determines whether or not the still image file of the image to be reproduced has been developed by the high image quality development unit 112. The determination method here is in accordance with the determination method in step S520 described with reference to FIG. 5, and thus detailed description thereof will be omitted.

  If the image has been developed by the high image quality developing unit 112 (YES in S603), the control unit 161 advances the process to step S620. This is because a still image file generated based on high-quality image information and stored in the storage medium 152 can be displayed with sufficient image quality even if it is read and enlarged.

  If the development at the high image quality development unit 112 is not performed (NO at S603), the control unit 161 advances the process to step S604, and thereby the development process by the high image quality development unit 112, that is, the follow-up development is executed. Will be. This is because a still image developed by the high image quality developing unit 112 can respond to a request for enlarged display with sufficient image quality. In steps S604 to S607, the same processing as steps S522 to S525 described with reference to FIG. 5 is performed, and thus detailed description thereof will be omitted.

  When the development processing in the high image quality development unit 112 in step S607 is completed, the high quality image information is output to the display processing unit 122 via the switch unit 121. Thereby, in step S608, the display processing unit 122 performs image display as shown in FIG. When the image display in step S608 is executed, the control unit 161 returns the process to step S601.

  Note that the high-quality development processing in steps S604 to S607 is assumed to occur at a timing at which follow-up development has not yet been performed, such as immediately after shooting. In the present embodiment, the chasing development is gradually ended when the imaging apparatus 100 is in an idle state with a small processing load, and the still image file that has been simply developed is replaced with the still image file that has been naturally developed with high image quality. As the still image file replacement progresses, the number of cases of executing the high-quality development processing in steps S604 to S607 decreases, so that a high-quality image can be quickly output for enlarged display, and operability is improved. Can be further increased.

  In the present embodiment, when a RAW file is held in the buffer 115, the process of step S605 can be skipped, so that an enlarged image can be displayed quickly. Therefore, it is preferable to adopt a configuration in which as many RAW files as possible are held in the buffer 115. Specifically, when the first display screen 700 in FIG. 7A and the second display screen 710 in FIG. 7B are made, a reduced image 701 and a standard image 711 are prepared in preparation for enlarged display. The RAW file corresponding to is read in advance from the storage medium 152 and moved to the buffer 115. Thus, by reading out and buffering the RAW file prior to the enlargement display instruction, when a display instruction of the enlargement image 721 such as the third display screen 720 in FIG. It is possible to improve the user's operational feeling and usability.

  Next, processing when the imaging apparatus 100 is in the moving image shooting mode will be described. FIG. 8 is a flowchart of processing when the imaging apparatus 100 is in the moving image shooting mode. Each process shown in FIG. 8 is performed by causing the CPU of the control unit 161 to expand a program stored in the ROM to the RAM and executing the program so that each block of the imaging apparatus 100 performs a predetermined operation under the control of the control unit 161. This is realized by executing the process.

  When the processing in the moving image shooting mode is started, in step S801, the control unit 161 determines whether or not the processing load on the imaging device 100 is small. When the processing load is small (YES in S801), the control unit 161 determines that the imaging apparatus 100 is in the idle state, proceeds to step S820, executes the processing in the idle state, and the processing load is not small. (NO in S801), the process proceeds to step S802. For example, when a video with a large number of pixels such as a horizontal resolution equivalent to 4000 pixels (4K) or 8000 pixels (8K) or a video with a high frame rate such as 120 frames per second (120P) is set, Since the processing load is high, the process always proceeds to step S802. On the other hand, if the number of pixels is less than the predetermined value or the frame rate is lower than the predetermined speed, for example, the process goes to step S820 halfway between the processing of the first frame and the second frame of the moving image. It is advanced.

  In step S802, the camera control unit 104 controls the operations of the imaging optical unit 101 and the imaging sensor 102 so as to perform moving image shooting under appropriate conditions. Specifically, the camera control unit 104 moves a lens included in the imaging optical unit 101 in accordance with a zoom or focus instruction from the user, and sets a readout area of the imaging sensor 102 in accordance with an instruction on the number of pixels to be captured. Further, based on the evaluation value and subject information respectively supplied from the evaluation value calculation unit 105 and the recognition unit 131, control such as focus adjustment and tracking for a specific subject is performed.

  In subsequent step S <b> 803, the sensor signal processing unit 103 performs signal processing for pixel repair on the electrical signal output from the imaging sensor 102. For example, interpolation processing using peripheral pixel values and subtraction processing of a predetermined offset value are performed on the values of missing pixels and low reliability pixels. In the present embodiment, the video information of a raw (undeveloped) moving image output from the sensor signal processing unit 103 is referred to as a “RAW image” as in the case of the still image shooting described above.

  In step S804, the simple developing unit 111 develops the RAW image. At this time, the control unit 161 sets the switch unit 121 so that the simple image information is output from the simple developing unit 111. Specifically, the simplified developing unit 111 simplifies by limiting the image size after development to, for example, 2 million pixels or less of HD video, and limiting or omitting noise removal and optical distortion correction to a limited process. To do. As a result, the development process can be speeded up, and high-speed shooting of an HD size moving image can be realized with a small circuit scale and low power consumption.

  The simple image information generated by the simple developing unit 111 is supplied to the evaluation value calculating unit 105. Accordingly, in step S805, the evaluation value calculation unit 105 calculates an evaluation value such as a focus state and an exposure state from the brightness value, contrast value, and the like included in the simple image information. Note that the evaluation value calculation unit 105 may acquire a RAW image before development processing and similarly calculate an evaluation value from the acquired RAW image. Further, the simple image information generated by the simple development unit 111 is supplied to the recognition unit 131. In step S806, the recognition unit 131 performs subject detection such as the presence / absence and position of a face, authentication of a specific person, and the like from the simple image information, and outputs the result as subject information.

  The simple image information generated by the simple developing unit 111 is supplied to the display processing unit 122. In step S807, the display processing unit 122 forms a display image from the acquired simple image information, and outputs and displays the display image on the display unit 123 or an external display device. The display image by the display unit 123 is used for confirmation display for the user to appropriately frame the subject in the moving image shooting mode. For example, unlike still image shooting, a display image during movie shooting is live for appropriately framing a subject not only before recording of the recorded movie (during standby) but also during recording (during REC). Used for view display.

  Note that an image may be displayed on another display device such as an external television from the display processing unit 122 via the external output terminal 124. In addition, the display processing unit 122 utilizes the evaluation value and the subject information supplied from the evaluation value calculation unit 105 and the recognition unit 131, respectively, to display the marking on the focused area on the display image and to the recognized face position. The frame display or the like may be performed.

  In step S808, the control unit 161 receives a recording start instruction from the user, and determines whether the moving image is being recorded (REC). If recording is in progress (YES in S808), control proceeds to step S809. If recording is not in progress, that is, if standby is in progress (NO in S808), processing returns to step S801.

  In step S809, the moving image compressing unit 142 sequentially acquires simple image information that is a captured moving image generated by the simple developing unit 111 with respect to a shooting target from the start of recording to the end of recording. On the other hand, compression processing is performed in units of frames. At that time, although not shown, audio information is acquired by a microphone (not shown) simultaneously with the shooting of the moving image, and the moving image compression unit 142 also performs compression processing on the audio information corresponding to the shooting moving image. It should be noted that the moving picture compression unit 142 applies MPEG-2, H.264, and simple image information and audio information. H.264 or H.264 A moving image file is generated by performing high-efficiency encoding processing (moving image compression) using a well-known moving image compression technique such as H.265. In subsequent step S <b> 810, the storage / playback unit 151 stores the moving image file output from the moving image compression unit 142 in the storage medium 152.

  A RAW image output from the sensor signal processing unit 103 for a shooting target from the start of recording to the end of recording (a RAW image in a period corresponding to the moving image to be recorded in step 808) is supplied to the RAW compression unit 113. ing. Therefore, in step S811, the RAW compression unit 113 performs high-efficiency encoding (RAW compression) on the RAW image acquired from the sensor signal processing unit 103 to generate a RAW file, and stores the generated RAW file in the buffer 115. .

  Note that high-efficiency encoding of a moving image RAW image performed by the RAW compression unit 113 can be performed by known techniques such as wavelet transform and differential encoding, as in the case of processing of a still image RAW image. Or reversible encoding. Alternatively, the RAW compression of the RAW compression unit 113 may be omitted, and the RAW image may be output through without being compressed. In the present embodiment, a RAW file that can be restored as a high-quality file that does not significantly impair the image information supplied from the sensor signal processing unit 103 is generated regardless of whether or not the RAW compression unit 113 performs high-efficiency encoding. To do.

  In step S812, the storage / playback unit 151 stores the RAW file output from the RAW compression unit 113 in the storage medium 152. Thereafter, the process returns to step S801. In steps S810 and S812, the storage / playback unit 151 may send and store the moving image file and / or the RAW file to the external storage via the communication terminal 154 by the communication unit 153.

  Here, the file structures of the moving image file and the moving image RAW file will be described. FIG. 9A is a diagram illustrating an example of a file structure of a moving image file, and FIG. 9B is a diagram illustrating an example of a file structure of a moving image RAW file. The moving image file 900 shown in FIG. 9A and the moving image RAW file 910 shown in FIG. 9B are stored in, for example, a predetermined storage area of the storage medium 152 by the storage / playback unit 151.

  The moving image file 900 includes a header portion 901, a metadata portion 902, and a compressed data portion 903. The header portion 901 includes an identification code indicating that this file is in a moving image file format. The metadata unit 902 includes file name information 904, development status 905, and shooting metadata 906. The file name information 904 is a file name of a moving image RAW file generated at the same time as the still image file. The development status 905 is information indicating that the moving image file has been developed by the simple development unit 111 or the high image quality development unit 112. The shooting metadata 906 includes the evaluation value calculated by the evaluation value calculation unit 105, the subject information detected by the recognition unit 131, and information at the time of shooting (for example, the lens type identification information of the imaging optical unit 101 and the sensor type of the imaging sensor 102). Identification information). Although not shown, the metadata unit 902 includes an identification code of the storage medium 152 in which a RAW file of a moving image generated simultaneously with the moving image file 900 is stored, path information of a stored folder, and the like. May further be included. The compressed data portion 903 includes high-efficiency encoded moving image and audio compressed data.

  The moving image RAW file 910 shown in FIG. 9B includes a header portion 911, a metadata portion 912, and a compressed data portion 913. The header portion 911 includes an identification code indicating that this file is in the RAW file format. The metadata unit 912 includes a file name 914, a development status 915, and shooting metadata 916. A file name 914 is a file name of a moving image file generated at the same time as the RAW file. The development status 915 is information indicating that the moving image file indicated by the file name 914 has been developed by the simple development unit 111 or the high image quality development unit 112. The shooting metadata 916 includes evaluation values calculated by the evaluation value calculation unit 105, subject information detected by the recognition unit 131, information at the time of shooting (for example, lens type identification information of the imaging optical unit 101 and sensor type of the imaging sensor 102). Identification information). Although not shown, the metadata unit 912 stores an identification code of the storage medium 152 in which the moving image file generated simultaneously with the moving image RAW file 910 is stored, path information of the stored folder, and the like. Further, it may be included. Further, the metadata unit 912 may store data obtained by extracting all or part of the moving image file generated at the same time as the moving image RAW file 910 (such as the first frame) and converting it into metadata. The compressed data portion 913 includes data obtained by performing high-efficiency encoding of a moving RAW image, but may be RAW image data that is not compressed.

  Note that the file structure shown in FIG. 9 is an example, and a configuration conforming to a standard such as DCF, AVCHD, MXF, or the like may be used.

  As described with reference to FIG. 8, the imaging apparatus 100 performs the RAW image development processing during the operation in the moving image shooting mode by the simple developing unit 111 and generates the moving image file during recording. A RAW file corresponding to is generated. The generated RAW file is a high-quality file that does not significantly impair the image information supplied from the sensor signal processing unit 103, but does not require development processing. Therefore, it is possible to realize development processing of moving image shooting with an increased number of pixels such as 4K and 8K and moving image shooting with an increased frame rate such as 120 frames per second (120P) with a small circuit scale and low power consumption. be able to.

  Next, processing when the imaging apparatus 100 is in the moving image playback mode will be described. FIG. 10 is a flowchart of processing when the imaging apparatus 100 is in the moving image playback mode. Each process shown in FIG. 10 is performed by causing the CPU of the control unit 161 to develop and execute a program stored in the ROM on the RAM, so that each block of the imaging apparatus 100 performs a predetermined operation under the control of the control unit 161. This is realized by executing the process.

  When the processing in the moving image reproduction mode is started, first, in step S1001, the control unit 161 determines whether or not the processing load on the imaging device 100 is small. When the processing load is small (YES in S1001), the control unit 161 determines that the imaging apparatus 100 is in the idle state, advances the process to step S1010, and executes the process in the idle state. If the processing load is not small (NO in S1001), control unit 161 advances the process to step S1002. For example, while waiting for a user operation such as a playback instruction, the processing load is low, and thus the process proceeds to step S1010. When the video playback is started by the user operation or during video playback, the process is The process proceeds to step S1002.

  In step S <b> 1002, the control unit 161 determines whether or not an instruction to pause playback (pause) is received from the user for the playback video. If there is no pause instruction (NO in S1002), control unit 161 advances the process to step S1003 in order to continue moving image reproduction. If there is a pause instruction (YES in S1002), the process advances to step S1020. .

  In step S1003, the storage / playback unit 151 reads a moving image file to be played back from the storage medium 152 or the like. In step S1004, the moving image extension unit 144 decodes and expands the moving image file frame by frame, and outputs the reproduced image of the moving image thus generated to the display processing unit 122. In subsequent step S1005, the display processing unit 122 performs a predetermined process on the reproduction image of the moving image acquired from the moving image expansion unit 144 and outputs the processed image to the display unit 123, whereby the moving image is reproduced and displayed on the display unit 123.

  When the process is advanced to step S1020 in response to the pause instruction in step S1002, the control unit 161 pauses the video being played back (displayed) and sets the frame at the pause position as a still picture. Display. When the still image is displayed in this way, it becomes easier to visually recognize the detailed image quality than when the image is moving. In addition, it is considered that it is easy to receive an enlarged display instruction during the pause. Therefore, in the present embodiment, when there is an instruction to pause, a frame having a higher image quality than that of the simple developed moving image file frame is displayed.

  For this purpose, in step S1020, the storage / playback unit 151 reads out a frame of a RAW image corresponding to a frame currently being paused and displayed from the RAW file of the moving image corresponding to the moving image file being reproduced. At this time, the storage / playback unit 151 reads from the buffer 115 if the RAW file to be played is buffered in the buffer 115, and reads from the storage medium 152 or the like if not buffered.

  In step S1021, the RAW decompression unit 114 decodes and decompresses the RAW file of the moving image read from the buffer 115 or the storage medium 152, restores the RAW image, and supplies the restored RAW image to the high image quality development unit 112. . In subsequent step S1022, the high image quality developing unit 112 develops the acquired RAW image with high image quality, and high quality image information is generated.

  The imaging apparatus 100 can capture, as a new still image file, a high-quality still image that has been developed from a moving image RAW file with high image quality and that corresponds to the frame being displayed by paused playback. . Therefore, in step S1023, the control unit 161 determines whether a user instruction to capture the display image at the reproduction pause position as a still image has been received. If the control unit 161 has not received a still image capture instruction (NO in step S1023), the control unit 161 advances the process to step S1005. As a result, the high-quality image information is supplied from the high-quality developing unit 112 to the display processing unit 122, and the display processing unit 122 outputs a high-quality still image corresponding to the frame being displayed to the display unit 123 by pausing playback. To do. Thereby, in step S1005, the frame of the simply developed moving image displayed on the display unit 123 is switched to the still image developed with high image quality.

  When the control unit 161 receives an instruction for capturing a still image (YES in S1023), the control unit 161 advances the process to step S1024. At that time, the high-quality image information is supplied from the high-quality image development unit 112 to the still image compression unit 141. Therefore, in step S1024, the still image compression unit 141 uses the high-quality image information acquired from the high-image quality development unit 112, and uses the well-known technique such as JPEG for the image information of the still image that has received the capture instruction. Efficiency coding processing (still image compression) is performed. The high-quality still image file generated in this way is supplied to the storage / playback unit 151.

  In step S1025, the storage / playback unit 151 stores the high-quality still image file acquired from the still image compression unit 141 in the storage medium 152 or the like. Thereafter, the control unit 161 advances the process to step S1005, whereby the frame of the simply developed moving image displayed on the display unit 123 is switched to a still image developed with high image quality.

  Note that the high-quality still image file generated by the still image compression unit 141 in step S1024 has the same configuration as the still image file 400 in FIG. In the metadata section 402, the RAW file name 404 is stored as the RAW file name 404. In addition, the shooting metadata 406 stores time information of a frame captured as a still image, and indicates a corresponding frame position of a moving image RAW file. When a high-quality still image file is generated in step S1024, the corresponding frame of the moving image RAW file may be extracted as a still image to create a new still image RAW file 410 to be paired. . When generating a RAW file of a still image, the still image file and the RAW file may be configured using the processing in steps S210 to S213 described with reference to FIG.

  Note that the image display in step S1005 is performed frame by frame, and the process returns to step S1001 in order to display the next frame during moving image playback. When the process proceeds from step S1001 to step S1010, the process according to the flowchart of FIG. 5 described above is executed.

  As described above, the imaging apparatus 100 can play back a moving image without delay using the moving image file stored in the storage medium 152 or the like when in the moving image playback mode. In addition, when video playback is paused, the paused display image can be replaced with a still image developed with high image quality from a RAW file, and the high-quality still image is captured as a still image file. be able to. These functions are highly convenient for the user.

  Further, it is assumed that the high-quality development processing in steps S1020 to S1025 in FIG. 10 occurs at a timing when the follow-up development is not executed, such as immediately after shooting. In the present embodiment, the chasing development of a RAW file of a moving image is gradually performed when the imaging apparatus 100 that is waiting for a user operation such as a period between shootings, a playback mode, an idle state, or the like has a relatively small processing load. finish. Thereby, the moving image file by simple development is naturally replaced with the moving image file by high image quality development. As the moving image file replacement progresses, the number of cases in which the processes in steps S1020 to S1025 occur decreases, so that a high-quality image can be output quickly, and the operability can be further improved.

  Although the present invention has been described in detail based on preferred embodiments thereof, the present invention is not limited to these specific embodiments, and various forms within the scope of the present invention are also included in the present invention. included. Furthermore, each embodiment mentioned above shows only one embodiment of this invention, and it is also possible to combine each embodiment suitably.

  The present invention can also be realized by executing the following processing. That is, software (program) that realizes the functions of the above-described embodiments is supplied to a system or apparatus via a network or various storage media, and a computer (or CPU, MPU, etc.) of the system or apparatus reads the program code. It is a process to be executed. In this case, the program and the storage medium storing the program constitute the present invention.

DESCRIPTION OF SYMBOLS 101 Imaging optical part 102 Imaging sensor 103 Sensor signal processing part 104 Camera control part 105 Evaluation value calculation part 110 Development part 111 Simple development part 112 High image quality development part 115 Buffer 122 Display processing part 123 Display part 131 Recognition part 151 Memory | storage reproduction | regeneration part 152 Storage medium

Claims (4)

An imaging apparatus comprising imaging means for generating a RAW image from a subject image,
First developing means for developing the RAW image;
A second developing unit that develops the RAW image with a developing process that is more burdensome than the first developing unit;
Storage means for storing the raw image, the image developed by the first developing means, and the image developed by the second developing means;
Extracting means for extracting subject information from the image developed by the first developing means;
Based on the subject information, a development priority for developing the RAW image stored in the storage unit by the second developing unit is obtained, and the RAW images stored in the storage unit in descending order of the development priority. Control means for developing the second developing means,
Display means for displaying an image;
When displaying the reduced image of one frame or the image of one frame on the display means, the image developed by the first developing means is displayed, and a part of the image of one frame is enlarged and displayed. An image pickup apparatus comprising: display processing means for displaying an image developed by the second developing means . Buffer means for temporarily storing the RAW image;
When the control unit displays the reduced image of the one frame or the image of the one frame on the display unit, the development processing by the second developing unit of the RAW image corresponding to the display image is not completed. The imaging apparatus according to claim 1 , wherein a RAW image corresponding to the display image is read from the storage unit and stored in the buffer unit. A method for controlling an imaging apparatus,
A generation step of generating a RAW image from the subject image at the time of shooting;
A first development step for performing development processing on the RAW image generated in the generation step;
A first storage step of storing the RAW image and the image developed by the first development step in a predetermined storage unit;
An extraction step for extracting subject information from the image developed in the first development step;
A second development step for developing the RAW image by a development process having a greater load on the process than the first development step;
Based on the subject information, a development priority for developing the RAW image stored in the storage unit in the second development step is obtained, and the RAW image stored in the storage unit in descending order of the development priority. A control step of developing in the second development step ;
A display step of displaying on the display means the image developed in the first development step or the second development step,
In the displaying step, when displaying a reduced image of one frame or an image of one frame, the image developed in the first developing step is displayed on the display means, and a part of the image of one frame is enlarged. And displaying the image developed in the second development step on the display means when displayed . In the control step, when the reduced image of the one frame or the image of the one frame is displayed on the display means in the display step, the developing process by the second developing step of the RAW image corresponding to the display image is completed. 4. The imaging apparatus control method according to claim 3, wherein if not, a RAW image corresponding to the display image is read from the storage unit and temporarily stored in a buffer unit.

Images