JP2017139618A - Image data generating apparatus, image data generating method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To generate image data capable of confirming gradation of an image of a high dynamic range (HDR) in detail even with a device incompatible with HDR.SOLUTION: A microcomputer (108) sets conversion tables corresponding to a plurality of small range areas narrower than the entire range area of an HDR so as to cover the entire range area of the HDR included in HDR image data. An image processing unit (103) converts HDR image data into a plurality of image data of SDR narrower than HDR on the basis of a plurality of small range areas set so as to cover the entire range area of HDR.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an image data generation device, an image data generation method, and a program for generating image data.

  An image data generation apparatus such as a digital camera or a digital video camera performs image processing on image data obtained by imaging a subject image or the like with an imaging element such as a CMOS sensor or a CCD, and digitally converting the image. In recent years, the performance of image pickup devices has improved, and image data with a high dynamic range can be generated. Hereinafter, the high dynamic range is expressed as “HDR”.

  However, the dynamic range that can be supported by the display device is limited, and when a display device that cannot display an HDR image is used, the HDR image cannot be displayed as it is in the dynamic range at the time of shooting. As a display device that cannot display an HDR image, a display device that supports only standard dynamic range image display is conceivable. Hereinafter, a standard dynamic range is expressed as “SDR”, and a display device supporting only SDR is expressed as “HDR non-compatible device”. For this reason, for example, when it is desired to display an HDR image on a device that does not support HDR, it is necessary to convert the HDR image data into image data having an SDR range width (for example, 8 bits).

  Japanese Patent Laid-Open No. 2004-151867 generates range-reduced image data by tone-compressing high-gradation image data, and generates additional data based on data lost due to tone compression by range reduction. Is disclosed.

JP 2014-220546 A

  However, for example, when an HDR non-compliant device is used when checking the gradation such as the color and brightness of an HDR image, the conventional technology displays a range-reduced image obtained by gradation-compressing high-gradation image data. Therefore, the gradation of the HDR image cannot be confirmed in detail.

  Accordingly, an object of the present invention is to generate image data that makes it possible to confirm in detail the gradation of an image with a high dynamic range even with a device that does not support the high dynamic range.

  The present invention provides setting means for setting a plurality of small range regions that are narrower than the entire range region of the first dynamic range so as to cover the entire range region of the first dynamic range of the first image data. Based on the plurality of small range regions set so as to cover the entire range region of the first dynamic range, the first image data of the first dynamic range is obtained from the first dynamic range. Conversion means for converting into a plurality of second image data having a narrow second dynamic range.

  According to the present invention, it is possible to generate image data that makes it possible to confirm in detail the gradation of an image with a high dynamic range even with a device that does not support the high dynamic range.

It is a figure which shows the internal structural example of the image data generation apparatus of embodiment. It is a figure which shows the internal structural example of an image process part. It is a figure used for outline | summary description of gradation conversion. It is a figure used for description of a gradation conversion table. It is a figure used for description of the example of a file structure of image data. It is a figure used for description of the structural example of the metadata of image data. It is a flowchart which shows the recording operation of an image data generation apparatus. It is a flowchart which shows the metadata setting operation | movement for every flame | frame. It is a figure used for description of an additional proxy production | generation area. It is a flowchart which shows additional proxy production | generation operation | movement. It is a figure used for description of the example of composition at the time of wireless transfer. It is a figure which shows the internal structural example of a portable terminal. It is a flowchart which shows the transfer operation | movement of compressed video data. It is a flowchart which shows the reproduction | regeneration operation | movement of a portable terminal. It is a figure which shows the UI example of a portable terminal. It is a figure used for description of another example of gradation conversion.

<Embodiment>
Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings.
FIG. 1 is a diagram illustrating an internal configuration example of an image data generation apparatus 100 according to the present embodiment. The image data generation device 100 of the present embodiment is assumed to be an HDR compatible camera such as a digital camera or a digital video camera that captures and records high dynamic range (HDR) image data.

  In FIG. 1, a microcomputer 108 (hereinafter referred to as a microcomputer 108) includes a CPU and the like, and controls the entire image data generation apparatus 100. The lens unit 101 is a correction lens group that has both a function of correcting the imaging position moved by the movement of the fixed lens group, the variable power lens group, the stop, and the variable power lens group for focusing, and the function of performing focus adjustment. It is comprised by. The lens unit 101 forms an object image on the image formation surface of the sensor 102.

  The sensor 102 is an image sensor and converts light into electric charges to generate an analog image signal. The image data generation device 100 according to the present embodiment can capture HDR images, and the sensor 102 is an image sensor that supports HDR. In the present embodiment, the sensor 102 includes an analog-digital conversion unit, and is a sensor that outputs 12-bit digital data as an example. In the image data generation device 100 of the present embodiment, as an example, it is assumed that moving image data is obtained. The sensor 102 is also provided with color filters corresponding to R (red), G (green), and B (blue) for generating a color image. Therefore, 12-bit R, G, and B image data are output from the sensor 102, respectively.

  In the image data generation device 100 of the present embodiment, 12-bit data can be processed in each of the components subsequent to the sensor 102. The image processing unit 103 performs predetermined image processing on the image data sent from the sensor 102. Image data after image processing by the image processing unit 103 is temporarily held in the RAM 111.

FIG. 2 is a diagram illustrating an internal configuration example of the image processing unit 103.
The processing unit 201 in FIG. 2 calculates a white balance gain value suitable for the image data sent from the sensor 102 and multiplies the image data. After that, the processing unit 201 converts each 12-bit R, G, and B image data into 12-bit Y, Cb, and Cr (YCbCr) image data, respectively, and a color information amount calculation unit 202. Send to. The color information amount calculation unit 202 calculates color information of Y, Cb, and Cr image data. The color information of the image data is a value obtained by dividing the sum of the pixel values of each pixel of the image data by the number of pixels (an average value of the pixel values of the image data). The color information amount calculation unit 202 outputs the calculated color information and Y, Cb, and Cr image data.

  FIG. 3A is a diagram showing how the luminance values (Y values) are particularly distributed in the 12-bit YCbCr image data. The vertical axis represents the number of pixels, and the horizontal axis represents the pixel values ( Y pixel value = luminance value). In the case of 12-bit YCbCr image data, the luminance value can take a value from 0 to 4095. In the case of the luminance distribution illustrated in FIG. 3A, a lot of luminance value data after 2048 is distributed, and therefore the average luminance value of the image data output from the image processing unit 103 is also a value after 2048. The value calculated by the color information amount calculation unit 202 is also 2048 or more. The average value of the pixel values calculated by the color information amount calculation unit 202 and 12-bit image data of YCbCr are temporarily stored in the RAM 111. The 12-bit image data of YCbCr output from the color information amount calculation unit 202 is also sent to the gradation conversion unit 205.

  The tone conversion unit 205 converts each 12-bit image data of YCbCr into 8-bit image data of YCbCr. As an example of gradation conversion by the gradation conversion unit 205, gradation conversion in luminance will be described with reference to FIGS. 3B and 3C, for example. FIG. 3B is a graph showing a correspondence relationship between a 12-bit pixel value (luminance value) before gradation conversion and an 8-bit pixel value (luminance value) after gradation conversion. In addition, the gradation conversion unit 205 according to the present embodiment can control how 12-bit data is mapped to 8-bit data.

  Here, 12-bit data can take values from 0 to 4095, while 8-bit data can take only values from 0 to 255. Therefore, in the gradation conversion, among the 12-bit image data of YCbCr, the pixel value having a luminance value of 0 to 15 is set to 0 for 8-bit data, and the pixel value having a luminance value of 16 to 31 is 8 bits. In this data, luminance conversion is performed so as to be 1 (the same applies hereinafter).

  The gradation conversion unit 205 performs gradation conversion (luminance conversion in the above example) using a table for converting 12-bit data to 8-bit data (hereinafter referred to as a gradation conversion table). . FIG. 3C shows an example of the gradation conversion table. The gradation conversion table shown in FIG. 3C is a table representing the correspondence between pixel values before gradation conversion and pixel values after gradation conversion. This gradation conversion table shows that every time the pixel value before conversion increases by 16, the pixel values 0 to 15 before gradation conversion are 0 after conversion, and the pixel values 16 to 31 before conversion are 1 after conversion. The table is such that the pixel value after conversion increases by one. Such a gradation conversion table is set for the gradation conversion unit 205 by the microcomputer 108 as setting means, for example. Each 8-bit YCbCr image data after gradation conversion by the gradation conversion unit 205 is temporarily stored in the RAM 111.

  The compression / decompression circuit A104 and the compression / decompression circuit B105 generate (encode) compressed video data by compressing image data temporarily stored in the RAM 111 according to a standard such as MPEG, and then hold the compressed video data in the RAM 111 again. . The compression / decompression circuit A104 and the compression / decompression circuit B105 also have a function of decompressing (decoding) compressed video data compressed according to a standard such as MPEG. Although not shown, the image data generation apparatus 100 according to the present embodiment also includes a microphone unit and a speaker. The compression / expansion circuit A104 and the compression / expansion circuit B105 perform compression / expansion of audio data together with image data. Also do. Then, the compression / expansion circuit A104 and the compression / expansion circuit B105 multiplex the audio data with the above-mentioned compressed data. In the following description, only compressed video data is taken as an example, and description of compressed audio data is omitted. Further, in the following description, when the compression / decompression circuit A104 and the compression / decompression circuit B105 are not distinguished, they are simply expressed as “compression / decompression circuit”.

  The ROM 110 is a flash ROM, and stores a program executed by the microcomputer 108. In addition, a partial area of the ROM 110 is used for backup and for holding the system state and the like. The RAM 111 is a volatile memory used as a work memory by the microcomputer 108, the image processing unit 103, the compression / decompression circuit, and the like.

  The memory card controller A112 controls writing and reading of data with respect to the memory card A114 in accordance with a format used in a computer such as a FAT (File Allocation Table) file system. Similarly, the memory card controller B113 controls writing and reading of data with respect to the memory card B115. The compressed video data stored in the RAM 111 is recorded on the memory card A 114 by the memory card controller A 112 and recorded on the memory card B 115 by the memory card controller B 113. Although details will be described later, in this embodiment, the memory card controller A112 writes and reads compressed video data of image data that has not undergone gradation conversion to the memory card A114. On the other hand, the memory card controller B113 writes and reads the compressed video data of the tone-converted image data with respect to the memory card B115. In the following description, when the memory card controller A 112 and the memory card controller B 113 are not distinguished, only “memory card controller” is described.

  The memory card A 114 and the memory card B 115 are recording media on which the above-described compressed video data and the like are recorded in a predetermined format such as a FAT file system. Further, the memory card A 114 and the memory card B 115 are recording media such as a removable SD card that can be detached from the image data generating apparatus 100, and can be mounted on a PC (personal computer) or the like in addition to the video camera. In the following description, when the memory card A 114 and the memory card B 115 are not distinguished, only “memory card” is described.

  The bus 117 is used for exchanging data between the blocks included in the image data generation apparatus 100 of the present embodiment. The external output unit 116 outputs image data or the like to an external device such as a display device under the control of the microcomputer 108. Image data output from the external output unit 116 is image data processed by the image processing unit 103 and stored in the RAM 111, image data read from the memory card, decoded by the compression / decompression circuit, and stored in the RAM 111, etc. It is. The wireless module 118 transfers the compressed video data and the like stored in the RAM 111 to an external device such as a portable terminal described later by wireless communication such as WiFi (registered trademark) or Bluetooth (registered trademark).

  Note that the external device connected via the external output unit 116 and the wireless module 118 is not only a device that can display an HDR image, but also an HDR non-compliant device that cannot display an HDR image. As a device that does not support HDR, for example, a display device or a portable terminal that supports only standard dynamic range (SDR) image display can be considered. In the present embodiment, it is assumed that an HDR non-compliant device that supports only SDR is, for example, a display device or a portable terminal that handles 8-bit data. Although details will be described later, the image data generation device 100 according to the present embodiment converts HDR image data into an SDR image when an HDR non-compliant device is connected via the external output unit 116 or the wireless module 118. It can be converted to data and output.

  The liquid crystal panel 107 is a panel capable of displaying a 12-bit HDR image. On the liquid crystal panel 107, an HDR image taken by the image data generation device 100 of the present embodiment, an image read from the memory card and decoded by the compression / decompression circuit, and the like are displayed. For example, the OSD unit 106 can superimpose information such as various setting menus, titles, and times on the HDR image data as display information for an on-screen display (OSD). In this case, an HDR image in which display information for OSD is combined is displayed on the liquid crystal panel 107.

  The operation switch group 109 is provided for a user to input an instruction or the like. In the operation switch group 109, for example, the user selects and instructs a camera mode mainly for photographing, a playback mode mainly for reproducing and displaying an image, a power-off mode for turning off the power, and the like. A switch or the like is provided. The operation switch group 109 includes a so-called touch panel.

  Next, with reference to FIGS. 4A to 4F, each gradation conversion table used when the gradation conversion unit 205 described above performs gradation conversion will be described. Each gradation conversion table described with reference to FIGS. 4A to 4F is a table having characteristics different from the gradation conversion table described with reference to FIG. 4A to 4F are examples, and more gradation conversion tables may be prepared. Each gradation conversion table described with reference to FIGS. 4A to 4F is a table for luminance (Y), and for CrCb, for example, the table shown in FIG. 3C is used.

  In the present embodiment, at the time of gradation conversion, any one of a plurality of gradation conversion tables as exemplified below is used depending on where the pixel values between 0 and 4095 before gradation conversion are weighted. It is done. In the present embodiment, among the pixel values 0 to 4095 before gradation conversion, a gradation conversion table weighted to the pixel value near the center, a gradation conversion table weighted to the vicinity of the upper limit of the pixel value, and the pixel value An example is a gradation conversion table weighted in the vicinity of the lower limit. Hereinafter, the gradation conversion table weighted to the pixel value near the center is the “middle conversion table”, the gradation conversion table weighted near the upper limit of the pixel value is the “high conversion table”, and the lower limit of the pixel value A gradation conversion table with weights in the vicinity is referred to as a “row conversion table”.

  FIG. 4D is an example of a middle conversion table, and FIG. 4A is a graph showing a correspondence relationship between pixel values before gradation conversion and pixel values after gradation conversion in the middle conversion table. The middle conversion table converts pixel values from 1024 to 3071 in the vicinity of the center among pixel values 0 to 4095 before gradation conversion into pixel values after gradation conversion with a resolution of 256. Specifically, pixel values 0 to 1023 before gradation conversion are converted to pixel value 0 after gradation conversion, and pixel values 3072 to 4095 before gradation conversion are converted to pixel value 255 after gradation conversion. The In this case, pixel values 0 to 1023 before gradation conversion are pixel values that are blackened after gradation conversion, and pixel values 3807 to 4095 before gradation conversion are pixel values that are overexposed after gradation conversion. Become. On the other hand, the pixel values 1024 to 3071 before gradation conversion are converted so that the pixel value after gradation conversion increases by 1 every time the pixel value before gradation conversion increases by 8. As described above, the middle conversion table is a predetermined small range region (pixel values 1024 to 3071 before gradation conversion) of all the dynamic range areas represented by pixel values 0 to 4095 before gradation conversion. This is a table for tone conversion with a resolution of 256. The middle conversion table converts the range of pixel values 1024 to 3071 before gradation conversion with a resolution of 256, whereas the gradation conversion table shown in FIG. The pixel values 0 to 4095 are tone-converted with a resolution of 256. For this reason, the image after the gradation conversion of the pixel values 1024 to 3071 before gradation conversion by the middle conversion table is performed more than when the gradation conversion table shown in FIG. 3C is used. High resolution image.

  FIG. 4E is an example of a high conversion table, and FIG. 4B is a graph showing a correspondence relationship between pixel values before gradation conversion and pixel values after gradation conversion in the high conversion table. The high conversion table converts the pixel values from 2047 to 4095, which are near the upper limit of the pixel value, among the pixel values 0 to 4095 before the gradation conversion into the pixel values after the gradation conversion with a resolution of 256. Specifically, pixel values 0 to 2047 before gradation conversion are converted to pixel values 0 after gradation conversion. In this case, pixel values 0 to 2047 before gradation conversion are pixel values that are blackened after gradation conversion. On the other hand, the pixel values 2048 to 4095 before gradation conversion are converted so that the pixel value after gradation conversion increases by 1 every time the pixel value before gradation conversion increases by 8. As described above, the high conversion table is a small range region (range of pixel values 2048 to 4095 before gradation conversion) among all the dynamic range areas represented by pixel values 0 to 4095 before gradation conversion. Is a table for gradation conversion with a resolution of 256. The high conversion table converts the range of pixel values 2048 to 4095 before gradation conversion with a resolution of 256, whereas the gradation conversion table of FIG. The pixel values 0 to 4095 are tone-converted with a resolution of 256. For this reason, the image after the gradation conversion of the pixel values 2048 to 4095 before gradation conversion by the high conversion table is performed more than when the gradation conversion table shown in FIG. High resolution image.

  FIG. 4F is an example of a row conversion table, and FIG. 4C is a graph showing a correspondence relationship between pixel values before gradation conversion and pixel values after gradation conversion in the row conversion table. The row conversion table converts pixel values from 0 to 2047, which are near the lower limit of pixel values, among pixel values 0 to 4095 before gradation conversion into pixel values after gradation conversion with a resolution of 256. Specifically, pixel values 2048 to 4095 before gradation conversion are converted to pixel values 255 after gradation conversion. In this case, pixel values 2048 to 4095 before gradation conversion are pixel values that are overexposed after gradation conversion. On the other hand, pixel values 0 to 2047 before gradation conversion are converted so that the pixel value after gradation conversion increases by 1 every time the pixel value before gradation conversion increases by 8. In this way, the low conversion table has a small range area (range of pixel values 0 to 2047 before gradation conversion) in the entire range area of the dynamic range represented by pixel values 0 to 4095 before gradation conversion. Is a table for gradation conversion with a resolution of 256. The row conversion table converts the range of pixel values 0 to 2047 before gradation conversion with a resolution of 256, whereas the gradation conversion table of FIG. The pixel values 0 to 4095 are tone-converted with a resolution of 256. For this reason, the image after the gradation conversion of the pixel values 0 to 2047 before gradation conversion by the row conversion table is performed more than in the case of using the gradation conversion table shown in FIG. High resolution image.

  In the following description, the original image data as it is without HDR conversion as described above is referred to as “master file”. Further, the image data after gradation conversion using the middle conversion table of FIG. 4D described above is referred to as “base proxy file”. Image data after gradation conversion using a gradation conversion table other than the middle conversion table is referred to as an “additional proxy file”. In the present embodiment, the gradation conversion tables other than the middle conversion table are, for example, the high conversion table in FIG. 4E and the low conversion table in FIG.

  FIG. 5 shows a file configuration example of image data generated by the image data generation apparatus 100 of the present embodiment and recorded on the memory card. The image data generation device 100 forms a folder 600 having the configuration shown in FIGS. 6A and 6B in the memory card.

  In the folder 600, master files 601, 607, base proxy files 602, 608, additional proxy files 603-606, 609, and the like are generated. The master file 601 is a file of original image data recorded without performing gradation conversion. The base proxy file 602 is a file of image data after gradation conversion is performed using the middle conversion table when the master file 601 is recorded.

  The additional proxy files 603 to 606 and 609 are image data files that have undergone gradation conversion different from the base proxy file 602. In the case of this embodiment, the additional proxy files 603 to 606, 609 are files in which image data subjected to gradation conversion using the above-described conversion table for high, conversion table for low, and the like are recorded. The additional proxy files 603 to 606 are files in which data after gradation conversion corresponding to different small range areas of the entire dynamic range of the master file 601 is recorded. For example, the additional proxy file 604 is a file in which data after gradation conversion different from the file 602 and the file 603 is recorded. Similarly, the additional proxy file 605 is a file in which data after gradation conversion different from the file 602 and the file 604 is recorded. The same applies to the additional proxy file 606. In the following description, an additional proxy file generated using the raw conversion table is referred to as an “additional proxy file (raw)”. Further, the additional proxy file generated using the high conversion table is referred to as “additional proxy file (high)”.

  The additional proxy files 601 to 606 are files related to the same clip. Next, when shooting is performed, a master file 607 and a base proxy file 608 are generated for the clip by the shooting. Depending on the clip, only the master file and the base proxy file may be generated, and the additional proxy file may not be generated. The case where the additional proxy file is not generated will be described later.

  Further, when the image data generating apparatus 100 according to the present embodiment records the folder 600 as shown in FIG. 5 on the memory card, the image data generating apparatus 100 generates management information as described below and adds it to the compressed video data. Record. FIGS. 6A and 6B are diagrams illustrating an example of metadata generated by the microcomputer 108 of the image data generation apparatus 100 as one piece of information in the management information. The metadata is included in the management information, added to the compressed video data, and recorded on the memory card. Note that FIGS. 6A and 6B show only management information examples when gradation conversion is performed on luminance.

The microcomputer 108 generates metadata 1320 for each clip shown in FIG. 6A and metadata 1340 for each frame shown in FIG. 6B as metadata.
First, the metadata 1320 for each clip shown in FIG. The metadata 1320 for each clip is data existing for one clip recorded on the memory card, and is included in the management information file and added to the image data and recorded. Among the pieces of information included in the metadata 1320, in the base proxy file information (BaseProxy), the file name (FileName) 1321 is the file name of the base proxy file, and is the file name on the file system. In the base proxy file information, the luminance range (Bright) 1322 is set with information corresponding to the gradation conversion table applied in the luminance range of the base proxy file. For example, in the case of gradation conversion using the middle conversion table shown in FIG. 4D, the luminance range is a range of pixel values 1024 to 3071.

  In the additional proxy file (low) information (LowProxy), the presence / absence presence / absence (Exist) 1323 is set to information indicating the presence / absence of the additional proxy file (low). For example, if there is even one frame having a luminance range near the lower limit of the pixel value, an additional proxy file (low) is generated later by gradation conversion using a low conversion table. Therefore, the presence / absence 1323 of the presence in this case is set as an additional proxy file (raw) “present (present)”. On the other hand, if there is no frame having a luminance range near the lower limit of the pixel value, an additional proxy file (low) will not be generated by tone conversion using a low conversion table later. Therefore, the presence / absence 1323 of the presence in this case is set as an additional proxy file (raw) “none” (none).

  In the additional proxy file (low) information, the luminance range (Bright) 1324 is set to information corresponding to the gradation conversion table applied in the luminance range of the additional proxy file (low). For example, in the case of gradation conversion using the low conversion table shown in FIG. 4F, the luminance range is a range of pixel values 0 to 2047. In the additional proxy file (row) information, the file name [] (File []) is set as many as the number of additional proxy files (row). A file name (FileName) 1325 is a file name on the file system of the additional proxy file (raw). The start frame number (StartFrameNo) 1326 sets the start frame number of the corresponding base proxy file. In the frame number (FrameNum) 1327, the number of frames of the additional proxy file (low) is set.

  In the additional proxy file (high) information (HighProxy), the presence / absence / existence (Exist) 1328 is set to information indicating the presence / absence of the additional proxy file (high). For example, if there is even one frame having a luminance range near the upper limit of the pixel value, an additional proxy file (high) is generated later by gradation conversion using the high conversion table. Therefore, the presence / absence 1328 in this case is set as an additional proxy file (high) “present (present)”. On the other hand, when there is no frame having a luminance range near the upper limit of the pixel value, an additional proxy file (high) is not generated by gradation conversion using the high conversion table later. For this reason, the presence / absence 1328 in this case is set to “none”.

  In the additional proxy file (high) information, the luminance range (Bright) 1329 is set with information corresponding to the gradation conversion table applied in the luminance range of the additional proxy file (high). For example, in the case of gradation conversion by the high conversion table shown in FIG. 4E, the luminance range is a range of pixel values 2048 to 4095. In the additional proxy file (high) information, the file name [] (File []) is set by the number of additional proxy files (high). A file name (FileName) 1330 is a file name on the file system of the additional proxy file (high). The start frame number (StartFrameNo) 1331 sets the start frame number of the corresponding base proxy file. The number of frames (FrameNum) 1332 sets the number of frames of the additional proxy file (high).

Next, the metadata 1340 for each frame shown in FIG. 6B will be described.
The metadata for each frame is metadata that exists for each frame constituting the clip, and is recorded in the main file, the base proxy file, and the additional proxy file, respectively. For example, a clip composed of 100 frames holds metadata for each frame in each of 100 frames.

  In the main file, a frame number (FrameNo) 1351 is a number incremented from 0 to each frame at the start of recording. Further, the minimum value (Min) 1352 is the minimum value of the luminance (Bright) of the corresponding frame, and the maximum value (Max) 1353 is the maximum value of the luminance of the corresponding frame.

  In the base proxy file, a frame number (FrameNo) 1361 is a number incremented from 0 to each frame at the start of recording. Presence / absence (Exist) 1362 is information “present” or “absent” set in the additional proxy file (low proxy) information (low proxy), and the minimum luminance value, maximum luminance value, and base proxy of the corresponding frame. Determined by the luminance range of the file. Similarly, presence / absence (Exist) 1363 is information “present” or “absent” set in the additional proxy file (High) information (HighProxy), and the luminance minimum value, luminance maximum value, and base of the corresponding frame Determined by the luminance range of the proxy file.

  Among the additional proxy file (low) and the additional proxy file (high), the frame number (FrameNo) 1371 is the frame number of the corresponding main file or base proxy file. Hereinafter, when the additional proxy file (low) and the additional proxy file (high) are described together, they are referred to as “additional proxy file (low / high)”.

  Next, FIG. 7 shows a flowchart of processing in the image data generation apparatus 100 of the present embodiment. Each process in the flowchart of FIG. 7 is realized by the microcomputer 108 of the image data generation apparatus 100 executing the image data generation program according to the present embodiment stored in the ROM 110. In the following description, steps S401 to S411 of each process in FIG. 7 are abbreviated as S401 to S411. Note that the image data generation device 100 according to the present embodiment includes image data that has undergone gradation conversion and image data that has not undergone gradation conversion from the image processing unit 103 to the RAM 111 each time the sensor 102 captures an image. Is output. In this state, the flowchart starts.

  In step S <b> 401, the microcomputer 108 sets the above-described middle conversion table for the gradation conversion unit 205 of the image processing unit 103. As a result, the gradation conversion unit 205 performs gradation conversion using the middle conversion table. Therefore, image data that has been subjected to gradation conversion by the middle conversion table is output to the RAM 111 together with image data that has not been subjected to gradation conversion. After S401, the microcomputer 108 proceeds to S402.

  In S <b> 402, the microcomputer 108 determines whether a recording start instruction is input from the user via the operation switch group 109 to the image data generation device 100. The microcomputer 108 continues the determination in S402 when the recording start instruction is not input, and proceeds to S420 when it is determined that the recording start instruction is input.

  In S420, the microcomputer 108 sets the metadata for each clip shown in FIG. 6A at the start of recording. Specifically, the microcomputer 108 sets the file name of the base proxy file 602 such as “Clip000_BaseProxy” in FIG. 5 as the file name 1321 of the base proxy file information in the management information file. Further, the microcomputer 108 sets information indicating the pixel values “1024 to 3071” of the above-described middle conversion table in the luminance range 1322 of the base proxy file information of the management information file. After S420, the microcomputer 108 proceeds to S403.

  In step S <b> 403, the microcomputer 108 controls the compression / decompression circuit A <b> 104 to compress the image data that has not been subjected to gradation conversion and output from the image processing unit 103 to the RAM 111. The image data compressed by the compression / decompression circuit A104 is written in the buffer area of the RAM 111 (hereinafter referred to as the compression / decompression circuit A buffer area) as the compressed video data of the master file. Similarly, the microcomputer 108 controls the compression / decompression circuit B105 to compress the image data after gradation conversion is performed by the middle conversion table output from the image processing unit 103 to the RAM 111. The image data compressed by the compression / expansion circuit B105 is written as compressed video data in a buffer area of the RAM 111 (hereinafter referred to as a compression / expansion circuit B buffer area). After S403, the microcomputer 108 proceeds to S404.

  In S404, the microcomputer 108 sets the metadata for each frame shown in FIG. Hereinafter, details of the processing of S404 will be described with reference to the flowchart of FIG. In the flowchart of FIG. 8, the processing steps S1400 to S1406 are abbreviated as S1400 to S1406.

  In the flowchart of FIG. 8, the microcomputer 108 first sets the frame number 1351 of the main file and the frame number 1361 of the base proxy file shown in FIG. Set. After S1401, the microcomputer 108 advances the process to S1402.

  In step S1402, the microcomputer 108 acquires the above-described color information of the corresponding frame calculated by the color information amount calculation unit 202 in the image processing unit 103. Then, the microcomputer 108 sets the minimum luminance value and maximum luminance value of the corresponding frame, which are obtained from the acquired color information, as the minimum value 1352 and maximum value 1353 respectively corresponding to the main file. After S1403, the microcomputer 108 proceeds to S1403.

  In step S1403, the microcomputer 108 determines whether or not the minimum luminance value of the corresponding frame obtained earlier is outside the luminance range of the base proxy file (specifically, whether or not it is less than the luminance range). If the microcomputer 108 determines in S1403 that the minimum luminance value is not less than the luminance range, the microcomputer 108 advances the process to S1405. On the other hand, if the microcomputer 108 determines in S1403 that the minimum luminance value is less than the luminance range, the microcomputer 108 advances the process to S1404. In step S <b> 1404, the microcomputer 108 sets the presence / absence of the presence of the additional proxy file (raw) 1362 as “existence” of the additional proxy file (raw). After S1404, the microcomputer 108 advances the process to S1405.

  In step S <b> 1405, the microcomputer 108 determines whether or not the maximum luminance value of the corresponding frame obtained earlier is outside the luminance range of the base proxy file (specifically, whether or not it exceeds the luminance range). If the microcomputer 108 determines in S1405 that the maximum luminance value does not exceed the luminance range, the microcomputer 108 ends the process of the flowchart of FIG. 8 and advances the process to S405 of FIG. On the other hand, if the microcomputer 108 determines in S1405 that the maximum luminance value exceeds the luminance range, the microcomputer 108 proceeds to S1406. In step S <b> 1406, the microcomputer 108 sets the presence / absence of an additional proxy file (high) 1363 to “existence” of the additional proxy file (high). After S1406, the microcomputer 108 ends the process of the flowchart of FIG. 8 and advances the process to S405 of FIG.

  As a specific example of the setting according to the flowchart of FIG. For example, when the minimum luminance value of the corresponding frame is 1100 and the maximum luminance value is 3000, the microcomputer 108 sets the presence / absence 1362 to the additional proxy file (low) “none” and the presence / absence 1363 to the additional proxy file (high) “none”. To "". Further, for example, when the minimum luminance value of the corresponding frame is 1100 and the maximum luminance value is 3100, the microcomputer 108 sets the presence / absence 1362 to the additional proxy file (low) “none” and the presence / absence 1363 to the additional proxy file (high). Set to “Yes”. Further, for example, when the minimum luminance value of the corresponding frame is 1000 and the maximum luminance value is 3100, the microcomputer 108 sets the presence / absence 1362 as an additional proxy file (low) “present” and the presence / absence 1363 as an additional proxy file (high). Set “Yes”.

  Returning to the flowchart of FIG. When the process proceeds to S405 in FIG. 7, the microcomputer 108 checks the buffer area for the compression / decompression circuit A in the RAM 111 and determines whether or not a certain amount of compressed video data is accumulated. If the microcomputer 108 determines in S405 that data has accumulated in the buffer area for the compression / decompression circuit A, the microcomputer 108 proceeds to S406. If the microcomputer 108 determines that data has not accumulated in the buffer area, The process proceeds to S407. In S <b> 406, the microcomputer 108 controls the memory card controller A <b> 112, and the compressed video data of the image that has not undergone gradation conversion and is stored in the buffer area for the compression / decompression circuit A of the RAM 111 with respect to the memory card A <b> 114. Write.

  Similarly, in S405, the microcomputer 108 confirms the buffer area for the compression / decompression circuit B in the RAM 111, and determines whether or not a certain amount of compressed video data is accumulated. Similarly, if the microcomputer 108 determines in S405 that data has accumulated in the buffer area for the compression / decompression circuit B, the microcomputer 108 proceeds to S406, and if it determines that data has not accumulated in the buffer area. The process proceeds to S407. In S <b> 406, the microcomputer 108 controls the memory card controller B <b> 113, and the compressed video data of the gradation-converted image stored in the compression / decompression circuit B buffer area of the RAM 111 with respect to the memory card B <b> 115. Write.

  After S406, the microcomputer 108 proceeds to S407. In step S <b> 407, the microcomputer 108 determines whether a recording stop instruction is input to the image data generation device 100 from the user via the operation switch group 109. If the microcomputer 108 determines in S407 that a recording stop instruction has been input, the microcomputer 108 proceeds to S408. If the microcomputer 108 determines that a recording stop instruction has not been input, the microcomputer 108 returns the process to S403.

  In step S408, the microcomputer 108 confirms the buffer area for the compression / decompression circuit A in the RAM 111, and determines whether or not the compressed video data is accumulated (remains). If the microcomputer 108 determines in S408 that data has accumulated in the buffer area for the compression / decompression circuit A, the microcomputer 108 proceeds to S409. If it determines that data has not accumulated in the buffer area, the microcomputer 108 proceeds to S410. To proceed. In step S409, the microcomputer 108 controls the memory card controller A112, and the compressed video data of the image that has not been subjected to gradation conversion and is stored in the buffer area for the compression / decompression circuit A of the RAM 111 with respect to the memory card A114. Write.

  Similarly, in S408, the microcomputer 108 confirms the compression / decompression circuit B buffer area of the RAM 111 and determines whether or not the compressed video data is accumulated. Similarly, if the microcomputer 108 determines in S408 that data has accumulated in the buffer area for the compression / decompression circuit B, the microcomputer 108 proceeds to S409, and if it determines that data has not accumulated in the buffer area. The process proceeds to S410. In S409, the microcomputer 108 controls the memory card controller B113, and the compressed video data of the image subjected to gradation conversion stored in the buffer area for the compression / decompression circuit B of the RAM 111 with respect to the memory card B115. Write.

  After S409, the microcomputer 108 proceeds to S410. In S410, the microcomputer 108 sets the metadata for each clip when recording is stopped. If the microcomputer 108 has set the additional proxy file (low) “present” even once as the metadata for each frame in S404, the presence / absence 1323 of the additional proxy file (raw) in the management information file is “present”. Set to. Further, for example, when the additional proxy file (low) is set to “none” in all frames, the microcomputer 108 sets the presence / absence 1323 of the additional proxy file (raw) in the management information file to “none”. . Further, the microcomputer 108 sets information “0-2047” as the luminance value in the luminance range 1324 of the additional proxy file (low) of the management information file.

  Similarly, the microcomputer 108 sets the presence / absence 1328 of the presence of the additional proxy file (high) in the management information file when the additional proxy file (high) is set to “present” even once as the metadata for each frame in S404. To "". Further, when the additional proxy file (high) is set to “none” in all frames, the microcomputer 108 sets the presence / absence 1328 of the presence of the additional proxy file (high) in the management information file to “none”. Further, the microcomputer 108 sets information “2048 to 4095” as the luminance value in the luminance range 1329 of the additional proxy file (high) of the management information file.

  After S410, the microcomputer 108 proceeds to S411. In S411, the microcomputer 108 writes the master file data stored in the RAM 111 to the memory card A 114 via the memory card controller A 112. Further, the microcomputer 108 writes the data of the base proxy file stored on the RAM 111 to the memory card B115 via the memory card controller B113. Thereafter, the microcomputer 108 ends the processing of the flowchart of FIG.

  Next, an operation in which the image data generation apparatus 100 according to the present embodiment generates an additional proxy file from a recorded main file will be described with reference to FIGS. 9 and 10. The image data generation device 100 may execute this process immediately after recording the main file, may be executed at an arbitrary timing designated by the user, or immediately before confirming a moving image on a portable terminal described later. May be executed. In the following description, tone conversion with respect to luminance is taken as an example.

  FIG. 9 is a diagram showing a luminance range for each frame and an additional proxy file generated therefrom. In FIG. 9, a delimiter 1531 indicated by a solid line is a delimiter in units of GOP (group of pictures) for each of a plurality of frames in MPEG, and a delimiter 1532 indicated by a dotted line is a frame delimiter. The frame groups 1501, 1502, 1503, and 1504 are composed of frames in which the pixel values are in the luminance range of 1024 to 3071. The frame groups 1511 and 1512 are composed of frames including a luminance range with pixel values of 0 to 1023. The frame groups 1521 and 1522 are composed of frames including a luminance range with pixel values of 3072 to 4095. Each frame includes a frame including a plurality of luminance ranges, such as frame 1 included in both the frame groups 1501 and 1511 as an example.

  The base proxy file 1505 is not generated at the timing shown in FIG. 9, but is generated at the same time as the main file recording described above, and is generated in the entire clip section. The additional proxy file (raw) is generated with a GOP unit including the frame groups 1511 and 1512 as a predetermined setting section. For example, the first additional proxy file (low) 1513 is generated within the GOP section including the frame group 1511, and the second additional proxy file (low) 1514 is generated within the GOP section including the frame group 1512. Generated. The first additional proxy file (high) 1523 is generated within the GOP section including the frame group 1521. In the case of the example of FIG. 9, since the frame group 1521 is included in two adjacent GOPs, the additional proxy file (high) 1523 is generated with two GOPs as predetermined setting sections. The second additional proxy file (high) 1524 is generated in the GOP section including the frame group 1522.

  FIGS. 10A to 10E are flowcharts for generating an additional proxy file from the main file recorded on the memory card. Each process in these flowcharts is realized by the microcomputer 108 of the image data generation device 100 executing the image data generation program according to the present embodiment stored in the ROM 110. In the following description, steps S1600 to S1645 in each flowchart are abbreviated as S1600 to S1645.

  First, the microcomputer 108 executes the processing from the flowchart of S1600 in FIG. FIG. 10A is a flowchart of an additional proxy file (low / high) generation process. In this embodiment, the GOP unit is a predetermined setting section when determining an additional proxy file (low / high).

  In step S1601, the microcomputer 108 executes an additional proxy file (raw) generation process. Detailed processing in S1601 will be described in S1610 of FIG. After S1601, the microcomputer 108 proceeds to S1602.

  In step S1602, the microcomputer 108 executes an additional proxy file (high) generation process. Detailed processing in S1602 will be described in S1630 in FIG. After S1602, the microcomputer 108 ends the process of generating the additional proxy file.

  Hereinafter, the process of S1610 of FIG. 10B will be described. S1610 in FIG. 10B relates to a process of searching for a luminance range for the target section of the entire clip as a target section, and subsequent generation processing of an additional proxy file (low). When proceeding to the processing of the flowchart of S1610, the microcomputer 108 proceeds to S1611.

  In step S <b> 1611, the microcomputer 108 sets the first GOP of the clip as a GOP that is a target when the luminance range is to be searched. After S1611, the microcomputer 108 proceeds to S1612. In S <b> 1612, the microcomputer 108 performs a luminance range search process for each frame in the GOP set in S <b> 1611. The detailed processing content of S1611 will be described with reference to FIG. After S1612, the microcomputer 108 proceeds to S1613.

  In S1613, the microcomputer 108 determines whether or not the processing in S1612 has been executed for all GOPs of the corresponding clip. If the microcomputer 108 determines in S1613 that execution of all GOP processes has been completed, the microcomputer 108 proceeds to S1615. If the microcomputer 108 determines that execution of all GOP processes has not been completed, the microcomputer 108 proceeds to S1614. .

  When the process proceeds to S1614, the microcomputer 108 sets the next GOP and returns the process to S1612. When the process proceeds to S1615, the microcomputer 108 generates an additional proxy file (low) for the additional proxy file (low) generation section in the corresponding clip set in S1612. Specifically, the microcomputer 108 sets a raw conversion table for the gradation conversion unit 205 of the image processing unit 103. Thereafter, the microcomputer 108 controls the memory card controller A112 to input the compressed video data of the image that has not been subjected to the gradation conversion and stored in the memory card A114 to the compression / decompression circuit A104. As a result, the compression / decompression circuit A104 performs a decoding process on the input compressed video data and outputs image data that has not undergone gradation conversion to the RAM 111. After that, the microcomputer 108 controls the image processing unit 103 to perform tone conversion using the row conversion table on the image data that has not been subjected to tone conversion output to the RAM 111. The tone-converted image data is converted into compressed video data by the compression / decompression circuit B105, and stored in the memory card B115 via the memory card controller B113.

  At this time, the microcomputer 108 sets the file name of the additional proxy file (raw) in the file name 1325 in the additional proxy file (raw) information of the management information file shown in FIG. Further, the microcomputer 108 sets the start frame number of the additional proxy file (raw) as the start frame number 1326 and sets the frame number as the frame number 1327. Further, the microcomputer 108 sets a frame number as metadata for each frame in the frame number 1371. After S1615, the microcomputer 108 ends the process of S1610 in FIG. 10B and advances the process to S1602 of FIG.

  Next, the process of S1620 of FIG. 10D showing details of S1612 of FIG. 10B will be described. In step S1620 in FIG. 10D, a search is performed for all frames in the GOP to determine whether or not there is a frame having a luminance range of pixel values 0 to 1023, and the corresponding GOP is added to the range of the additional proxy file (low). It is determined whether or not to set. In the flowchart of S1620, the microcomputer 108 first performs the process of S1621.

  In S1621, the microcomputer 108 sets the first frame in the GOP as a frame to be searched when the luminance range is to be searched. After S1621, the microcomputer 108 proceeds to S1622. In S1622, the microcomputer 108 determines whether or not the minimum luminance value acquired from the metadata of the corresponding frame is less than the luminance range of the base proxy file acquired from the management information file. If the microcomputer 108 determines in S1622 that the minimum luminance value is less than the luminance range, the microcomputer 108 proceeds to S1623. If the microcomputer 108 determines that the minimum luminance value is not less than the luminance range, the microcomputer 108 proceeds to S1624.

  When the process proceeds to S <b> 1623, the microcomputer 108 sets the corresponding GOP in the range of the additional proxy file (low). After S1623, the microcomputer 108 ends the process of S1620 in FIG. Thereby, the process of S1600 in FIG.

  When the process proceeds to S1624, the microcomputer 108 determines whether or not the process of S1622 has been executed for all frames in the GOP. If the microcomputer 108 determines in S1624 that the process of S1622 has not been executed for all frames in the GOP, the process proceeds to S1625. On the other hand, if it is determined in S1624 that the process of S1622 has been executed for all frames in the GOP, the process of S1620 in FIG. When the process proceeds to S1625, the microcomputer 108 sets the frame to be processed as the next frame, and then returns the process to S1622.

  Next, details of S1630 in FIG. 10C will be described. S1630 in FIG. 10C relates to a process of searching for a luminance range for the target section of the entire clip as a target section and a subsequent generation process of an additional proxy file (high).

  In the flowchart of S1630, the microcomputer 108 first performs the process of S1631. In S1631, the microcomputer 108 sets the head GOP of the clip as the GOP to be searched when the luminance range is to be searched. After S1631, the microcomputer 108 proceeds to S1632. In S1632, the microcomputer 108 performs a luminance range search process for each frame in the GOP set in S1631. The detailed processing content of S1632 will be described in S1640 of FIG. After S1632, the microcomputer 108 advances the process to S1633.

  In step S1633, the microcomputer 108 determines whether the processing in step S1632 has been executed for all GOPs of the corresponding clip. If the microcomputer 108 determines in S1633 that execution of processing for all GOPs has been completed, the microcomputer 108 proceeds to S1635. If the microcomputer 108 determines that execution of processing for all GOPs has not been completed, the processing proceeds to S1634. To proceed.

  In S1634, the microcomputer 108 sets the next GOP and returns the process to S1633. On the other hand, in S1635, the microcomputer 108 generates an additional proxy file (high) for the generation section of the additional proxy file (high) in the corresponding clip set in S1632. Specifically, the microcomputer 108 sets a high conversion table for the gradation conversion unit 205 of the image processing unit 103. Thereafter, the microcomputer 108 controls the memory card controller A112 to input the compressed video data of the image that has not been subjected to gradation conversion and stored in the memory card A114 to the compression / decompression circuit A104. The compression / decompression circuit A104 performs a decoding process on the input compressed video data, and outputs image data on which gradation conversion has not been performed to the RAM 111. Further, the microcomputer 108 controls the image processing unit 103 and performs gradation conversion using the high conversion table on the image data that has been output to the RAM 111 and has not been subjected to gradation conversion. The tone-converted image data is converted into compressed video data by the compression / decompression circuit B105 and stored in the memory card B115 via the memory card controller B113.

  At this time, the microcomputer 108 sets the file name of the additional proxy file (high) in the file name 1330 in the additional proxy file (high) information of the management information file in FIG. Further, the microcomputer 108 sets the start frame number to the start frame number 1331 and sets the frame number to the frame number 1332. Further, the microcomputer 108 sets a frame number as metadata for each frame in the frame number 1371. After S1635, the microcomputer 108 ends the process of S1630 of FIG. 10C, and ends the process of the flowchart of FIG.

  Next, the processing of S1640 of FIG. 10E showing details of S1632 of FIG. 10C will be described. S1640 in FIG. 10E searches for all frames in the GOP for a frame having a luminance range of pixel values 2048 to 4095, and sets the corresponding GOP in the range of the additional proxy file (high). Decide whether or not. In the flowchart of S1640, the microcomputer 108 first performs the process of S1641.

  In S1641, the microcomputer 108 sets the first frame in the GOP as a frame to be searched for the luminance range from now on. After S1641, the microcomputer 108 proceeds to S1642. In S1642, the microcomputer 108 determines whether the maximum luminance value acquired from the metadata of the corresponding frame exceeds the luminance range of the base proxy file acquired from the management information file. If the microcomputer 108 determines in S1642 that the maximum luminance value exceeds the luminance range, the microcomputer 108 proceeds to S1643. If the microcomputer 108 determines that the maximum luminance value does not exceed the luminance range, the microcomputer 108 proceeds to S1644.

  When the process proceeds to S1643, the microcomputer 108 sets the corresponding GOP in the range of the additional proxy file (high). After S1643, the microcomputer 108 ends the process of S1640 of FIG. Thereby, the process of S1600 in FIG.

  When the process proceeds to S1644, the microcomputer 108 determines whether or not the process of S1642 has been executed for all frames in the GOP. If the microcomputer 108 determines in S1644 that the process of S1642 has not been executed for all frames in the GOP, the process proceeds to S1645. On the other hand, if it is determined in S1644 that the process of S1642 has been executed for all frames in the GOP, the process of S1640 in FIG. If the process proceeds to S1645, the microcomputer 108 sets the frame to be processed as the next frame, and then returns the process to S1642.

  Next, the moving image obtained by encoding the compressed video data on the memory card by the processing of the flowcharts of FIGS. 7 and 10A to 10E is displayed on the display screen of the mobile terminal, for example. An operation when the user confirms the content of the gradation of the moving image will be described.

  In FIG. 11, a video camera which is an example of the image data generation device 100 of the present embodiment wirelessly transmits moving image data to a mobile terminal 702 such as a tablet terminal, and displays it on the screen of the mobile terminal 702. The example of a structure in case a moving image is displayed is shown. In the example of FIG. 11, the portable terminal 702 can display only the compressed video data (SDR 8-bit compressed video data) subjected to gradation conversion by the image data generation device 100 of the present embodiment. It is assumed that the device is not compatible. For this reason, it is assumed that the portable terminal 702 cannot display compressed video data (HDR 12-bit compressed video data) that has not undergone gradation conversion.

  The image data generation device 100 and the portable terminal 702 can be connected by wireless communication. The image data generation device 100 transmits the compressed video data after gradation conversion stored in the memory card B115 to the portable terminal 702 by wireless communication. Thereby, the portable terminal 702 can display the compressed video data after gradation conversion.

FIG. 12 is a diagram illustrating an internal configuration example of the mobile terminal 702 according to the present embodiment.
In the portable terminal 702 shown in FIG. 12, a microcomputer (microcomputer 1001) is composed of a CPU or the like and controls the entire portable terminal 702. A bus 1006 is used for exchanging data between blocks included in the mobile terminal 702. The operation switch group 1002 is, for example, a touch panel provided for a user to input an instruction or the like. The operation switch group 1002 includes a power switch and the like.

  The wireless module 1007 is connected to, for example, the image data generation device 100 of the present embodiment by wireless communication, and receives the compressed video data transmitted from the image data generation device 100 and the compressed audio data multiplexed thereon. The received compressed video data and compressed audio data are temporarily stored in the RAM 111 under the control of the microcomputer 1001 and then stored in the recording medium 1005. The compressed video data and the compressed audio data stored in the recording medium 1005 are sent to the compression / decompression circuit 1008 under the control of the microcomputer 1001.

  The compression / decompression circuit 1008 decodes the compressed video data and compressed audio data in the RAM 1004. The compression / decompression circuit 1008 also has a function of performing encoding. Image data and audio data decoded by the compression / decompression circuit 1008 are temporarily stored in the RAM 1004. The decoded image data is read from the RAM 1004 under the control of the microcomputer 1001, sent to the liquid crystal panel 1010 via the OSD unit 1009, and displayed. The decoded audio data is output as audio from a speaker or the like (not shown).

  The liquid crystal panel 1010 is a panel that can display an SDR image. For example, the OSD unit 1009 can superimpose information such as various setting menus and time as display information for OSD on the SDR image data. In this case, an SDR image in which display information for OSD is combined is displayed on the liquid crystal panel 1010.

  A ROM 1003 is a flash ROM and stores a program executed by the microcomputer 1001. Further, a partial area of the ROM 1003 is used for backup and for holding the system state and the like. A RAM 1004 is a volatile memory used as a work memory by the microcomputer 1001, the compression / decompression circuit 1008, and the like.

  FIG. 13 is a flowchart of processing in which the image data generating apparatus 100 transfers the compressed video data after gradation conversion as described above to the portable terminal 702. It is assumed that the image data generation device 100 and the portable terminal 702 are connected via the wireless module 118 and the wireless module 1007 before the processing of the flowchart of FIG. Each process in the flowchart of FIG. 13 is realized by the microcomputer 108 of the image data generation apparatus 100 executing a program stored in the ROM 110. In the following description, steps S801 to S805 of each process in FIG. 13 are abbreviated as S801 to S805. In the flowchart of FIG. 13 and FIG. 14 to be described later, only compressed video data is taken as an example. A description of an example of compressed audio data is omitted.

  In step S <b> 801, the microcomputer 108 determines whether a compressed video data transfer instruction is input to the image data generation device 100 from the user via the operation switch group 109. In step S801, the microcomputer 108 returns to the determination in step S801 if a transfer instruction for compressed video data is not input, and proceeds to step S802 if it is determined that a transfer instruction is input.

  In step S <b> 802, the microcomputer 108 controls the memory card controller B <b> 113 to read out the compressed video data compressed after gradation conversion using the above-described middle conversion table from the memory card B <b> 115 and temporarily store it in the RAM 111. . Thereafter, the microcomputer 108 transfers the compressed video data read from the RAM 111 to the portable terminal 702 via the wireless module 118. After S802, the microcomputer 108 advances the process to S803.

  In step S <b> 803, the microcomputer 108 controls the memory card controller B <b> 113 to read out the compressed video data compressed after gradation conversion using the above-described raw conversion table from the memory card B <b> 115 and temporarily store it in the RAM 111. . Thereafter, the microcomputer 108 transfers the compressed video data read from the RAM 111 to the portable terminal 702 via the wireless module 118. After S803, the microcomputer 108 proceeds to S804.

  In step S <b> 803, the microcomputer 108 controls the memory card controller B <b> 113 to read out the compressed video data compressed after gradation conversion using the above-described high conversion table from the memory card B <b> 115 and temporarily store it in the RAM 111. . Thereafter, the microcomputer 108 transfers the compressed video data read from the RAM 111 to the portable terminal 702 via the wireless module 118. After S804, the microcomputer 108 proceeds to S805.

  In step S <b> 805, the microcomputer 108 controls the memory card controller B <b> 113 to read the above-described management information file data from the memory card B <b> 115 and temporarily store it in the RAM 111. Thereafter, the microcomputer 108 transfers the management information file data read from the RAM 111 to the portable terminal 702 via the wireless module 118. When the process of S805 ends, the microcomputer 108 ends the process of the flowchart of FIG.

  FIG. 14 shows a flowchart of processing until the portable terminal 702 receives and reproduces (displays) the compressed video data transferred from the image data generating apparatus 100. Each process in the flowchart of FIG. 14 is realized by the microcomputer 1001 of the portable terminal 702 executing a program stored in the ROM 1003. In the following description, steps S1101 to S1109 of each process in FIG. 14 are abbreviated as S1101 to S1109.

  Here, the microcomputer 1001 of the portable terminal 702 monitors whether or not compressed video data has been transmitted from the image data generation device 100 via the wireless module 1007. When the microcomputer 1001 determines that the data has been transmitted, the microcomputer 1001 controls the wireless module 1007 to store the compressed video data sent from the image data generation device 100 in the RAM 1004 and then to the recording medium 1005. . At this time, the image data generating apparatus 100 also sends the data of the management information file corresponding to the compressed video data to be transmitted together with the compressed video data. The microcomputer 1001 also stores the management information file data in the recording medium 1005. The received compressed video data is data of at least the above-described base proxy file, and may further include an additional proxy file (low / high). The flowchart of FIG. 14 is a flowchart of the reproduction process when a base proxy file and an additional proxy file (low / high) are received. The process of the flowchart of FIG. 14 starts when the user inputs a playback instruction for compressed video data (additional file playback instruction) via the operation switch group 1002.

  In S1101, the microcomputer 1001 determines a playback start frame number. The microcomputer 1001 normally starts playback from the first frame of the clip for which playback is instructed. However, for example, when a resume playback instruction is input, the microcomputer 1001 starts playback from the frame next to the frame where playback was stopped during the previous playback. After S1101, the microcomputer 1001 advances the process to S1102.

  In step S1102, the microcomputer 1001 acquires information indicating the presence / absence of an additional proxy file (low) or an additional proxy file (high) from the management information file. Note that the microcomputer 1001 determines whether or not there is an additional proxy file (low) including the playback start frame number from the start frame number 1326 and the number of frames 1327 of the additional proxy file (low) in the management information file described above. Good. Similarly, the microcomputer 1001 may determine the presence / absence of an additional proxy file (high) including the playback start frame number from the start frame number 1331 and the number of frames 1332 of the additional proxy file (high) in the management information file. . After S1102, the microcomputer 1001 advances the process to S1103.

  In step S1103, the microcomputer 1001 causes the OSD unit 1009 to generate, for example, a button icon indicating the presence / absence of an additional proxy file (low / high) based on the information on the presence / absence of the additional proxy file (low / high) determined in step S1102. . The button icon generated by the OSD unit 1009 is displayed on the screen of the liquid crystal panel 1010. As an example, the microcomputer 1001 controls the OSD unit 1009 to display a button icon in, for example, white when it is determined that the additional proxy file (low / high) is “present”, while it is determined to be “none”. In this case, the button icon is displayed in gray, for example. At this time, the microcomputer 1001 controls the OSD unit 1009 to simultaneously generate a button icon representing the base proxy file and display it on the liquid crystal panel 1010. Since the base proxy file includes all frames of the corresponding clip, for example, it is always displayed as a white button icon. After S1103, the microcomputer 1001 advances the process to S1104.

  FIG. 15 is a diagram illustrating a display example of icons and the like displayed on the screen of the liquid crystal panel 1010 in S1103. In the example of FIG. 15, a low button 1201, a middle button 1202, and a high button 1203 are displayed as button icons. The Low button 1201 is an icon corresponding to an additional proxy file (low), the Middle button 1202 is an icon corresponding to a base proxy file, and the High button 1203 is an icon corresponding to an additional proxy file (high). The Low button 1201 is displayed in white when the additional proxy file (low) is “present” and can be selected by the user, while when it is “not present”, it is displayed in gray and cannot be selected. Similarly, the high button 1203 is displayed in white when the additional proxy file (high) is “present” and can be selected by the user, while when it is “not present”, it is displayed in gray and cannot be selected. The The Middle button 1202 is always displayed in white and can be selected by the user.

  In step S1104, the microcomputer 1001 determines whether an instruction to play an additional proxy file (low) is input from the user via the operation switch group 1002. In step S1104, the microcomputer 1001 determines that an instruction to play an additional proxy file (raw) has been input, and if there is an additional proxy file (raw), the microcomputer 1001 advances the processing to step S1105. On the other hand, if the microcomputer 1001 determines in step S1104 that an instruction to play an additional proxy file (raw) has not been input, or if there is no additional proxy file (raw), the microcomputer 1001 advances the processing to step S1106.

  In S1105, the microcomputer 1001 reproduces the additional proxy file (raw). Specifically, the microcomputer 1001 controls the recording medium 1005 to input the stored compressed video data of the additional proxy file (raw) to the compression / decompression circuit 1008. The compression / decompression circuit 1008 performs a decoding process on the input compressed video data, and outputs the decoded image data to the RAM 1004. Thereafter, the microcomputer 1001 sends the image data temporarily stored in the RAM 1004 to the liquid crystal panel 1010 via the OSD unit 1009 and causes the liquid crystal panel 1010 to display the image data. As a result, an image of the additional proxy file (raw), that is, an image whose gradation has been converted by the raw conversion table is displayed on the screen of the liquid crystal panel 1010. After S1105, the microcomputer 1001 advances the process to S1109.

  On the other hand, in S1106, the microcomputer 1001 determines whether or not a reproduction instruction for the additional proxy file (high) has been input. In step S <b> 1106, the microcomputer 1001 determines that an instruction to play an additional proxy file (high) has been input, and if the additional proxy file (high) is “Yes”, the microcomputer 1001 advances the process to step S <b> 1107. On the other hand, if the microcomputer 1001 determines in S1106 that an instruction for reproducing the additional proxy file (high) has not been input, or if the additional proxy file (high) is “none”, the microcomputer 1001 proceeds to S1108. Proceed.

  In step S1107, the microcomputer 1001 plays the additional proxy file (high). Specifically, the microcomputer 1001 controls the recording medium 1005 to input the stored compressed video data of the additional proxy file (high) to the compression / decompression circuit 1008. The compression / decompression circuit 1008 performs a decoding process on the input compressed video data, and outputs the decoded image data to the RAM 1004. Thereafter, the microcomputer 1001 sends the image data temporarily stored in the RAM 1004 to the liquid crystal panel 1010 via the OSD unit 1009 and causes the liquid crystal panel 1010 to display the image data. As a result, an image of the additional proxy file (high), that is, an image subjected to gradation conversion by the high conversion table is displayed on the screen of the liquid crystal panel 1010. After S1107, the microcomputer 1001 advances the process to S1109.

  In S1108, the microcomputer 1001 reproduces the base proxy file. Specifically, the microcomputer 1001 controls the recording medium 1005 to input the stored compressed video data of the base proxy file to the compression / decompression circuit 1008. The compression / decompression circuit 1008 performs a decoding process on the input compressed video data, and outputs the decoded image data to the RAM 1004. Thereafter, the microcomputer 1001 sends the image data temporarily stored in the RAM 1004 to the liquid crystal panel 1010 via the OSD unit 1009 and causes the liquid crystal panel 1010 to display the image data. As a result, an image of the base proxy file, that is, an image whose gradation has been converted by the middle conversion table is displayed on the screen of the liquid crystal panel 1010. After S1108, the microcomputer 1001 advances the process to S1109.

  In step S <b> 1109, the microcomputer 1001 determines whether a reproduction stop instruction is input to the mobile terminal 702 from the user via the operation switch group 1002. If the microcomputer 1001 determines in S1109 that the reproduction stop instruction has not been input, the microcomputer 1001 returns the process to S1102, and if it determines that the reproduction stop instruction has been input, the microcomputer 1001 ends the process of the flowchart in FIG.

  As described above, according to the present embodiment, the image data generation device 100 generates compressed video data obtained by performing gradation conversion on the image data before gradation conversion using a plurality of gradation conversion tables, and compressing the compressed video data. The video data and the management information file are transferred to the mobile terminal 702. The portable terminal 702 can display an image obtained by decoding compressed video data obtained by gradation conversion using the plurality of gradation conversion tables. The plurality of gradation conversion tables are tables that generate SDR image data for each of a plurality of small range regions corresponding to the SDR dynamic range from the entire HDR dynamic range region of the HDR image. ing. In the portable terminal 702, the user can select which of the compressed video data subjected to gradation conversion using the plurality of gradation conversion tables is to be reproduced and displayed.

  Therefore, even if the portable terminal 702 is a device that does not support HDR, for example, capable of displaying SDR images, the user can select and play back the compressed video data that has undergone gradation conversion using a plurality of gradation conversion tables. The gradation of the image before the tone conversion can be confirmed. In particular, the image data generation apparatus 100 generates and transfers only necessary sections as an additional proxy file (low) or an additional proxy file (high), except for the base proxy file that has been tone-converted using the middle conversion table. . Therefore, according to the present embodiment, the user can check a wide range of gradations while ensuring the continuous reproduction by the base proxy file and suppressing the additional proxy file generation time, transfer time, and data size. Become.

  In the above description, the gradation conversion table is switched to a conversion table for high, middle, and low, and 12-bit image data is converted to 8 bits using these three types of gradation conversion tables. An example is given. As another example, among the 12-bit image data before gradation conversion, for example, gradation conversion that cuts the upper 4 bits, gradation conversion that cuts the upper 2 bits and lower 2 bits, and a floor that cuts the lower 4 bits Tone conversion may be performed to generate 8-bit image data.

  Specifically, as shown in FIG. 16A, in the gradation conversion for cutting the upper 4 bits, the upper 4 bits B8 among the bits B0 to B11 of the 12-bit image data 901 before the gradation conversion. ~ B11 is cut. The remaining 8-bit B0 to B7 image data 904 is the data after gradation conversion. In the gradation conversion that cuts the lower 2 bits and the upper 2 bits, among the bits B0 to B11 of the 12-bit image data 901 before the gradation conversion, the upper 2 bits B0 and B1 and the lower 2 bits B10 and B11 are cut. Is done. The remaining 8-bit B2 to B9 image data 903 is the data after gradation conversion. In the gradation conversion for cutting the lower 4 bits, the lower 4 bits B0 to B3 are cut out of the bits B0 to B11 of the 12-bit image data 901 before the gradation conversion. The remaining 8-bit image data 902 of B4 to B11 is used as data after gradation conversion.

  FIG. 16B shows pixel values before gradation conversion when the image data 902 is generated by gradation conversion that cuts the lower 4 bits of the image data 901 before gradation conversion in FIG. It is a graph which shows the relationship with each pixel value after gradation conversion. As shown in FIG. 16B, the gradation conversion for cutting the lower 4 bits is performed by dividing the pixel value before gradation conversion by 16 and discarding the remainder as the pixel value after gradation conversion. It becomes processing to do. When gradation conversion for cutting the lower 4 bits is performed, the number of steps representing the resolution of the gradation of the image is reduced from 16 steps to one step, so the resolution for the gradation of the image is reduced to 1/16. . That is, in the case of 12-bit image data 901 before gradation conversion, the resolution representing gradation is 4095 steps, whereas in the 8-bit image data 902 after gradation conversion that cuts the lower 4 bits, The resolution representing gradation is 255 steps. However, the gradation-converted image data 902 that cuts the lower 4 bits has a resolution of 255 representing all gradation ranges that the image data 901 before gradation conversion has, although the resolution representing gradation is lowered. The data covers the resolution of the step. Therefore, it can be said that the gradation conversion that cuts the lower 4 bits is a conversion that is suitable for roughly grasping all gradations of the image before gradation conversion.

  FIG. 16C shows a state before gradation conversion when the image data 903 is generated by gradation conversion that cuts the upper 2 bits and the lower 2 bits of the image data 901 before gradation conversion in FIG. It is a graph which shows the relationship between each pixel value and each pixel value after gradation conversion. As shown in FIG. 16C, the gradation conversion for cutting the upper 2 bits and the lower 2 bits is performed by dividing the pixel value before gradation conversion by 4 and dividing the remainder by 4 again. Processing is performed so as to obtain a later pixel value. When gradation conversion that cuts the upper 2 bits and lower 2 bits is performed, the number of steps representing the resolution of the gradation of the image is changed from 4 steps to 1 step, so the resolution for the gradation of the image is reduced to ¼. Will be reduced. Further, the image data 903 after gradation conversion for cutting the upper 2 bits and the lower 2 bits is a partial area (specifically, among all gradations included in the image data 901 before gradation conversion. This is data that can express gradation only in a quarter area). However, in the case of gradation conversion that cuts the upper 2 bits and the lower 2 bits, a partial area (1/4 area) of all gradations included in the image data 901 before gradation conversion is as follows. The gradation is represented with a resolution of 255 steps. That is, in the gradation conversion that cuts the upper 2 bits and the lower 2 bits, the lower 4 bits are cut for some areas (1/4 area) of all the gradations of the image before gradation conversion. Gradation can be expressed with a finer resolution than in the case of gradation conversion.

  FIG. 16D shows each pixel value before gradation conversion when the image data 904 is generated by gradation conversion that cuts the upper 4 bits of the image data 901 before gradation conversion in FIG. It is a graph which shows the relationship with each pixel value after gradation conversion. As shown in FIG. 16D, the gradation conversion for cutting the upper 4 bits is a process in which the remainder obtained by dividing the pixel value before gradation conversion by 16 is used as the pixel value after gradation conversion. . For this reason, when gradation conversion that cuts the upper 4 bits is performed, the number of steps representing the gradation of the image is the same as that before and after gradation conversion. Is maintained before and after tone conversion. However, the image data 904 after gradation conversion that cuts the upper 4 bits is a partial area (specifically, 1/16 of all gradations included in the image data 901 before gradation conversion). This is data that can represent gradation only in (region). In the case of gradation conversion that cuts the upper 4 bits, gradation is expressed with a resolution of 255 steps in the 1/16 area. That is, in the gradation conversion for cutting the upper 4 bits, the upper 2 bits and the lower 2 bits are cut for a part of the entire range area (1/16 area) of the image before gradation conversion. The gradation can be expressed with a more detailed resolution than in the case of gradation conversion.

  For this reason, for example, when an image is displayed on a non-HDR compatible device, the user can roughly check the gradation for all gradations of the image before gradation conversion. It can be seen that this is an image after gradation conversion with bits cut. On the other hand, an image after gradation conversion with the upper 2 bits and lower 2 bits cut and an image after gradation conversion with the upper 4 bits cut, for example, when displayed on a non-HDR compatible device, Only part of the tonality is displayed. However, when the image is confirmed by a device that does not support HDR, the user needs to use the image data 903 after the gradation conversion that cuts the upper 2 bits and the lower 2 bits, and the image after the gradation conversion that cuts the upper 4 bits. The gradation can be confirmed more widely than the data 904. On the other hand, when an image is confirmed by a device that does not support HDR, the user needs to convert the image data 904 after the gradation conversion that cuts the upper 4 bits into the image data after the gradation conversion that cuts the upper 2 bits and the lower 2 bits. The gradation can be confirmed in more detail than the image data 903.

  For this reason, for example, when recording compressed video data subjected to gradation conversion, the image data generation device 100 generates compressed video data of image data subjected to gradation conversion for cutting the lower 4 bits, It is recorded on the memory card together with the image data before gradation conversion. When transferring the compressed video data, the image data generating apparatus 100 first generates the compressed video data of the image data after gradation conversion with the lower 4 bits cut, which is generated and recorded at the time of recording. Send to. In addition, when image data after gradation conversion in which the lower 4 bits are cut is being transmitted, the image data generation device 100 determines that the upper 2 from the image data before gradation conversion recorded in the memory card. Image data after gradation conversion in which bits and lower 2 bits are cut is generated. Then, the image data generation device 100 generates compressed video data from the image data after gradation conversion in which the upper 2 bits and the lower 2 bits are cut, and transmits the compressed video data to the portable terminal 702. Further, when transmission of the image data after gradation conversion with the upper 2 bits and lower 2 bits cut is performed, the image data generation device 100 stores the image data before gradation conversion recorded in the memory card. To generate image data after gradation conversion in which the upper 4 bits are cut. Then, the image data generation device 100 generates compressed video data from the image data after gradation conversion in which the upper 4 bits are cut, and transmits the compressed video data to the portable terminal 702.

  As described above, the image data generating apparatus 100 first sends the image data after gradation conversion with the lower 4 bits cut, and then sends the image data after gradation conversion with the upper 2 bits and the lower 2 bits cut, Finally, the image data after gradation conversion with the upper 4 bits cut is sent. As a result, the user of the portable terminal 702 can first check the overall gradation of the image data, and then gradually check the detailed gradation of the image data.

<Other embodiments>
The present invention supplies a program that realizes one or more functions of the above-described embodiments to a system or apparatus via a network or a storage medium, and one or more processors in a computer of the system or apparatus read and execute the program This process can be realized. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

  The above-described embodiments are merely examples of implementation in carrying out the present invention, and the technical scope of the present invention should not be construed as being limited thereto. That is, the present invention can be implemented in various forms without departing from the technical idea or the main features thereof.

  DESCRIPTION OF SYMBOLS 100 Image data generation apparatus, 103 Image processing part, 104 Compression / decompression circuit A, 105 Compression / decompression circuit B, 106 OSD part, 107 Liquid crystal panel, 108 Microcomputer, 109 Operation switch group, 110 ROM, 111 RAM, 112 Memory card controller A , 113 Memory card controller B, 114 Memory card A, 115 Memory card B, 116 External output unit, 118 Wireless module

Claims (12)

Setting means for setting a plurality of small range areas narrower than the entire range area of the first dynamic range so as to cover the entire range area of the first dynamic range included in the first image data;
Based on the plurality of small range areas set to cover the entire range area of the first dynamic range, the first image data of the first dynamic range is narrower than the first dynamic range. Conversion means for converting into a plurality of second image data having a second dynamic range;
An image data generation device comprising: The setting means includes
Setting a predetermined small range region in the entire range region of the first dynamic range of the first image data;
When the first image data has a pixel value outside the range of pixel values corresponding to the predetermined small range area, a small range area including a pixel value outside the range is further set. The image data generation device according to claim 1. The setting means includes
A range region near the center of all the range regions of the first dynamic range is set as the predetermined small range region;
3. The image data according to claim 2, wherein a small range region near the lower limit or near the upper limit is set as a small range region including a pixel value outside the range among all the range regions of the first dynamic range. Generator. The setting means includes
When the maximum pixel value of the first image data is a value that exceeds the range of pixel values corresponding to the predetermined small range region, a small range region including a pixel value outside the range is selected. , Set as a small range area near the upper limit,
When the minimum pixel value of the first image data is less than the pixel value range corresponding to the predetermined small range region, a small range region including a pixel value outside the range is selected. The image data generation device according to claim 3, wherein the image data generation device is set as a small range region near the lower limit. The setting means includes
When there is a frame having a value in which the maximum value of the pixel value exceeds the range of the pixel value corresponding to the small range region near the center in a predetermined setting section for each of a plurality of frames of the first image data that is a moving image. Sets a small range area near the upper limit for all frames of the predetermined setting section,
In the predetermined setting section of the first image data, when there is a frame whose value is less than the pixel value range corresponding to the small range region near the center, the predetermined pixel value The image data generation apparatus according to claim 4, wherein a small range area in the vicinity of the lower limit is set for all frames in the setting section. The setting means includes
Generating metadata indicating which region in the entire range region of the first dynamic range is the small range region;
The image data generation apparatus according to claim 1, wherein the metadata is added to the second image data. The setting means includes
If the first image data is video data, generate metadata indicating information indicating from which frame of the video the second image data was generated,
The image data generation apparatus according to claim 1, wherein the metadata is added to the second image data. The setting means includes
When the first image data is moving image data, for each frame of the moving image, the second image data in the small range region near the lower limit and the second image data in the small range region near the upper limit , Generate metadata that indicates the presence or absence of each,
The image data generation apparatus according to claim 3, wherein the metadata is added to second image data of the frame.   The setting means includes the small range area represented by cutting lower bits from a plurality of bits representing the pixel value of the first image data in correspondence with the first dynamic range, and the first range. A plurality of small range regions represented by cutting higher bits from a plurality of bits representing pixel values of image data, and a plurality of pixels representing pixel values of the first image data in correspondence with the first dynamic range The image data generation apparatus according to claim 1, wherein the small range area expressed by cutting upper bits and lower bits from the bits is set.   The second image data generated from the small range area represented by cutting the lower bits is first transmitted to an external device, and then the upper bits and the lower bits are cut and displayed. The second image data generated from the small range area to be transmitted to the external device, and finally the second image data generated from the small range area represented by cutting the upper bits is The image data generation apparatus according to claim 9, further comprising a control unit configured to transmit to an external device. Setting a plurality of small range areas narrower than the entire range area of the first dynamic range so that the setting means covers the entire range area of the first dynamic range of the first image data;
Based on the plurality of small range regions set so as to cover the entire range region of the first dynamic range, conversion means converts the first image data of the first dynamic range to the first dynamic range. Converting to a plurality of second image data having a second dynamic range narrower than the dynamic range;
An image data generation method comprising:   The program for functioning a computer as each means of the image data generation apparatus of any one of Claims 1 thru | or 10.

Images