JP2024001606A - Imaging device, method of controlling imaging device, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To output image data that is not RAW data such that display information such as icons and menu screens can be checked on a receiving side according to an operation state, even in the setting of outputting RAW data.

SOLUTION: An imaging device includes output means that output RAW data or development data generated by processing the RAW data to an external device, and control means that outputs the RAW data by the output means when an operation state regarding image recording is a recording state in the setting of outputting the RAW data, and controls development data to be output by the output means when the development data is not in the recording state.

SELECTED DRAWING: Figure 1

COPYRIGHT: (C)2024,JPO&INPIT

Description

The present invention relates to an imaging device, a method of controlling the imaging device, and a program.

An image captured by an imaging device may be output to an external device using an external output terminal of the imaging device, and the captured image may be checked on an external monitor or recorded on the external monitor itself. At this time, the output format of the image data to be externally output is specified on the imaging device side, or information on the receiving device is acquired and the output format is changed in accordance with a format that can be received by the receiving device. Patent Document 1 describes a technique for transmitting image data captured by an imaging device to a receiving device via an HDMI (registered trademark) transmission path as RAW data without converting it into YCC format image data. .

Japanese Patent Application Publication No. 2021-150791

However, in Patent Document 1, if the receiving device supports RAW data and outputting RAW data is selected in the HDMI output selection menu, the imaging device always outputs RAW data regardless of the operating state etc. is output to the receiving device via the HDMI transmission path. When this RAW data is output, display information such as icons and menu screens cannot be output to the receiving device, so the receiving device that receives the RAW data cannot display the icons, menu screens, etc. and cannot confirm the information.

The present invention makes it possible to output image data that is not RAW data, so that display information such as icons and menu screens can be checked on the receiving side, depending on the operating state, even if RAW data is set to be output. The purpose is to

The imaging device according to the present invention includes an output unit for outputting RAW data or developed data generated by developing the RAW data to the outside, and an operation related to image recording when the RAW data is set to be output. The apparatus is characterized by comprising a control means for controlling the RAW data to be output by the output means if the state is in the recording state, and to output the developed data by the output means if the state is not the recording state.

According to the present invention, even if the setting is to output RAW data, it is possible to output image data that is not RAW data, depending on the operating state, so that display information such as icons and menu screens can be checked on the receiving side. It becomes possible to do this.

1 is a diagram showing an example of the configuration of an imaging device. 5 is a flowchart showing processing of the imaging device in Embodiment 1. FIG. FIG. 3 is a diagram for explaining RAW data. FIG. 3 is a diagram for explaining details of RAW data. FIG. 3 is a diagram for explaining an example of on-screen display. FIG. 3 is a diagram for explaining a memory area for HDMI output. FIG. 3 is a diagram for explaining a state in which RAW data is stored in a memory area for HDMI output. FIG. 3 is a diagram for explaining a state in which RAW data is stored in a memory area for HDMI output. 3 is a diagram for explaining a display example in the first embodiment. FIG. It is a diagram showing an example of the configuration of a receiving device. 7 is a flowchart illustrating an example of processing of the imaging device in Embodiment 2. FIG. 7 is a diagram for explaining a display example in Embodiment 2. FIG. 7 is a flowchart illustrating an example of processing of a receiving device in Embodiment 2. FIG.

Embodiments of the present invention will be described below with reference to the drawings.

<Embodiment 1>
FIG. 1 is a block diagram showing a configuration example of an imaging device 100.
The lens unit 101 is an optical system that includes a fixed lens group for condensing light, a variable magnification lens group, an aperture, a correction lens group, and the like. The correction lens group is a lens group that has both the function of correcting the imaging position moved by the movement of the variable magnification lens group and the function of adjusting focus. A subject image is formed by the lens unit 101 on the imaging plane of an image sensor 102, which will be described later. The lens unit 101 is detachable from the imaging device 100.

The image sensor 102 is, for example, an image sensor such as a CCD image sensor or a CMOS image sensor, and converts light into charge to generate an image signal. The imaging signal generated by the image sensor 102 is output to, for example, the image processing unit 103. The image sensor is a so-called dual pixel, in which every pixel on the imaging surface is composed of a pair of light-receiving elements, and a pair of optical images formed by a microlens in each pixel can be photoelectrically converted by the pair of light-receiving elements. You may use the type.

The image processing unit 103 converts the image signal output from the image sensor 102 into RAW data (RAW image). The image processing unit 103 also performs RAW development processing such as interpolation processing and image quality adjustment processing on the converted RAW data to generate development data corresponding to the RAW data. In this embodiment, it is assumed that the development data is YCC4:2:2 format image data. That is, the RAW data of this embodiment is image data before YCC4:2:2 format image data is converted. RAW data (RAW image) and development data (YCC4:2:2 format image data) obtained by the image processing unit 103 are stored in the RAM 111.

The display resizing unit 104 performs resizing processing on the image data stored in the RAM 111 to generate display image data. The display resizer 104 stores the generated display image data in the RAM 111. The recording resizing unit 105 performs resizing processing on the image data stored in the RAM 111 to generate recording image data. The recording resizing unit 105 stores the generated recording image data in the RAM 111.

The on-screen display (OSD) generation unit 106 generates OSD data related to on-screen displays (OSD) such as icons and menu screens. The OSD generation unit 106 stores the generated OSD data in the RAM 111. The OSD data includes OSD data such as various setting menus, titles, and time. The OSD data stored in the RAM 111 is, for example, combined with the display image data stored in the RAM 111 and displayed on the display section 107 or outputted to the outside from the external output section 115. The display unit 107 is a display member for displaying display image data and OSD. The display unit 107 is, for example, a liquid crystal panel.

A microcomputer (hereinafter also referred to as microcomputer) 108 controls the entire imaging apparatus 100. The operation switch group 109 is an operation member for the user to input operations. The operation switch group 109 also includes a switch for selecting one of a camera mode for photographing, a playback mode for playback, and a power-off mode for turning off the power.

The ROM (Read Only Memory) 110 is, for example, a flash ROM, and stores programs executed by the microcomputer 108 and the like. Further, a part of the ROM 110 is used for backup purposes and to hold the state of the system. A RAM (Random Access Memory) 111 is a volatile memory used as a work memory by the microcomputer 108, the image processing unit 103, the compression/expansion unit 114, and the like.

The memory card controller 112 records the moving image data generated by the compression/expansion unit 114 and output to the RAM 111 on the memory card 113 in accordance with a format compatible with computers such as the FAT file system. The memory card 113 is a recording medium that is removably attached to the imaging device 100, and can be attached to a computer or the like in addition to the imaging device 100. The compression/decompression unit 114 encodes (for example, performs MPEG compression) the image data stored in the RAM 111 to generate moving image data, and outputs the moving image data to the RAM 111 .

The external output unit 115 outputs the image data etc. output to the RAM 111 by the image processing unit 103 to the outside. The external output unit 115 is, for example, an interface that complies with the HDMI standard, the SDI standard, and the like. The external output unit 115 is capable of outputting image data using signals of standards such as 4K60P and 2K60P. The bus 116 is a bus that exchanges data between each part of the imaging device 100.

Next, with reference to FIG. 2, a process in which the imaging apparatus 100 in the first embodiment outputs image data obtained by imaging will be described. In this embodiment, the imaging device 100 includes an interface that complies with the HDMI standard as the external output unit 115, and outputs image data to the outside using a signal that complies with the HDMI standard (hereinafter also referred to as an HDMI signal). do. Further, in this embodiment, it is assumed that image data in YCC4:2:2 format is generated as development data. FIG. 2 is a flowchart showing the processing of the imaging device 100 in the first embodiment. Each process in the flowchart of FIG. 2 is controlled by the microcomputer 108 of the imaging device 100 executing a program stored in the ROM 110.

In step S201, the microcomputer 108 determines whether a menu operation using the operation switch group 109 has been performed. This menu operation is an operation for setting the operation of the imaging device 100, such as setting the resolution of the video signal captured by the image sensor 102 or setting the bit rate for encoding by the compression/expansion unit 114. It is. Also, through this menu operation, settings for HDMI RAW mode, on-screen display settings, recording command settings, etc., which will be described later, are also performed. If the microcomputer 108 determines that a menu operation has been performed (YES in step S201), the microcomputer 108 stores information set by the menu operation as mode information in the RAM 111, and proceeds to step S202. If the microcomputer 108 determines that no menu operation has been performed (NO in step S201), the process advances to step S203.

In step S202, the microcomputer 108 performs mode setting processing and controls each functional unit in the imaging apparatus 100 so as to transition to the mode or state set in step S201.

In step S203, the microcomputer 108 refers to information held in the RAM 111 and determines whether HDMI connection has been completed. The HDMI connection is a state in which HDMI connection processing, which will be described later, has been completed, and information on this state is held in the RAM 111 through the processing in step S205. If the microcomputer 108 determines that the HDMI connection has been completed (YES in step S203), the process advances to step S206. If the microcomputer 108 determines that the HDMI connection has not been completed (NO in step S203), the process advances to step S204.

In step S204, the microcomputer 108 controls the external output unit 115 to check whether the hot plug detection signal (HPD) according to the HDMI standard is detected and whether the signal line of the TMDS signal according to the HDMI standard is pulled up. Determine. If the microcomputer 108 determines that the hot plug detection signal is detected and the signal line of the TMDS signal is pulled up (YES in step S204), the microcomputer 108 determines that there is an HDMI connection and proceeds to step S205. In other cases (NO in step S204), the microcomputer 108 determines that there is no HDMI connection, and ends the process of FIG. 2.

In step S205, the microcomputer 108 performs HDMI connection processing. In the HDMI connection process, the microcomputer 108 controls the external output unit 115 to acquire the EDID (Extended Display Identification Data) of the sink device connected to the external output unit 115 via HDMI. The EDID is information on the sink device connected via the external output unit 115, and is composed of data of a video format supported by the sink device, proprietary data of the vendor, and the like. The microcomputer 108 stores the acquired EDID in the RAM 111.

In step S206, the microcomputer 108 determines whether the imaging device 100 is in camera mode. Camera mode means that various signal processing is performed on image data that is focused by the lens unit 101 and captured by the image sensor 102, and is recorded on the memory card 113, displayed on the display unit 107, or displayed on the external output unit 115. This is the mode for outputting to. In addition to the camera mode, the imaging device 100 also has a playback mode in which image data recorded on the memory card 113 is displayed on the display unit 107 or output to the external output unit 115. If the microcomputer 108 determines that the imaging device 100 is in camera mode (YES in step S206), the process advances to step S207. On the other hand, if the microcomputer 108 determines that the imaging device 100 is not in the camera mode (NO in step S206), the microcomputer 108 ends the process of FIG. 2.

In step S207, the microcomputer 108 refers to the mode information held in the RAM 111, controls the image sensor 102, etc. according to the mode information, and captures RAW data (RAW image) related to the subject image. The captured RAW data is held in the RAM 111.

In step S208, the microcomputer 108 refers to the mode information held in the RAM 111 and determines whether the imaging device 100 is in the HDMI RAW mode. The HDMI RAW mode is a mode in which the imaging device 100 outputs RAW data captured in step S207 and subjected to γ processing in step S209 to an external sink device via the external output unit 115. If the microcomputer 108 determines that the mode is HDMI RAW mode (YES in step S208), the process advances to step S209. If the microcomputer 108 determines that the mode is not HDMI RAW mode (NO in step S208), the process advances to step S211.

In step S209, the microcomputer 108 controls the image processing unit 103 to perform γ processing on the RAW data captured in step S207.

In step S210, the microcomputer 108 controls the image processing unit 103 to write the RAW data subjected to the γ processing in step S209 to the memory area for HDMI output (VRAM for HDMI output) in the RAM 111.

Here, the RAW data to be written to the RAM 111 will be explained with reference to FIGS. 3 and 4. In FIG. 3A, 300 indicates the entire RAW data written to the RAM 111 in step S210. The RAW data 300 is composed of data of an effective pixel area (Effective Pixel Area) 301 and data of an additional pixel area (Additional Pixel Area) 302. As shown in FIG. 3(a), the surplus pixel area 302 is an area in which several pixels are added to the upper, lower, left, and right sides of the effective pixel area 301. Used when developing edges. Because of this configuration, the data written in step S210 is the sum of the effective pixel area and the surplus pixel area, as shown in FIG. 3(b). For example, in the case of 4K RAW data, data for a total of 4120x2176 pixels is written to the RAM 111, which is the effective image area of 4096x2160 pixels, 12 pixels of the left and right surplus pixel areas, and 8 pixels of the top and bottom surplus pixel areas. .

FIG. 4 is a diagram for explaining details of the RAW data written to the RAM 111 in step S210. In FIG. 4, as shown at 400, the RAW data is configured in an R/Gr/Gb/B Bayer array. The 4K RAW data has a Bayer array in which 2060 pieces (4120 pixels as pixel data) are arranged in the horizontal direction as shown in 401 and 1088 pieces (2176 pixels as pixel data) in the vertical direction as shown in 402. It is something. Note that the data size is 4120 (horizontal) x 2176 (vertical) x 12 (bit depth) ÷ 8 (bit to byte) = 13447680 bytes.

Returning to FIG. 2, in step S211, the microcomputer 108 controls the image processing unit 103 to perform RAW data correction processing on the RAW data captured in step S207. This RAW data correction processing includes pre-development lens correction processing (peripheral light amount correction processing, lateral chromatic aberration correction, etc.) and white balance processing. Data used for pre-development lens correction processing is stored in advance in the ROM 110 for each type of lens, and the microcomputer 108 uses this data and the type of lens unit 101 attached to the imaging device 100 to perform development processing. Determine parameters for front lens correction processing.

In step S212, the microcomputer 108 controls the image processing unit 103 to perform development processing on the image data subjected to the correction processing in step S211. This development processing includes processing such as debayer, γ processing, and color fringing correction. Regarding the color fringing correction, the microcomputer 108 determines the parameters of the correction process based on data stored in the ROM 110 in advance and the type of the lens unit 101 attached to the imaging device 100. By performing the development process, the RAW data becomes data in the YCC4:2:2 format. Image data (development data) in the YCC4:2:2 format after the development process is stored in the RAM 111.

In step S213, the microcomputer 108 controls the image processing unit 103 to perform development data correction processing such as distortion correction on the YCC4:2:2 format image data generated in step S212 and stored in the RAM 111. conduct. Regarding this distortion correction as well, the microcomputer 108 determines distortion correction parameters based on data stored in the ROM 110 in advance and the type of the lens unit 101 attached to the imaging device 100. The post-development correction data (YCC4:2:2 format) corrected in step S213 is stored in the RAM 111.

In step S214, the microcomputer 108 refers to the mode information held in the RAM 111 and determines whether the imaging device 100 is in the HDMI RAW mode. If the microcomputer 108 determines that the mode is HDMI RAW mode (YES in step S214), the process advances to step S215. If the microcomputer 108 determines that the mode is not HDMI RAW mode (NO in step S214), the process advances to step S217.

In step S215, the microcomputer 108 refers to the mode information held in the RAM 111 and determines whether the imaging device 100 is in the on-screen display mode. The on-screen display mode is based on the mode information and operating status held in the RAM 111, and displays information about the imaging device, such as shooting, settings, and operating status, in the form of icons, menu screens, character strings (text format), etc. This is a mode in which the image is displayed superimposed on the image.

FIGS. 5(a) to 5(c) are diagrams illustrating examples of on-screen display in which information is displayed superimposed on an image. Various types of display information 501 to 507 are displayed superimposed on an image 500 captured by the image sensor 102. In the example shown in FIGS. 5(a) to 5(c), 501 is information on the usable time of a power source (such as a battery) that supplies power to the imaging device 100, and 502 is information on the remaining record of the memory card 113. This is information about available time. Further, 503 is information on the recording operation state of the imaging apparatus 100, and 504 is an audio level meter indicating the state of audio input. Further, 505, 506, and 507 are information on the current shutter speed, ISO value, color information, and bit depth of the imaging apparatus 100, respectively. Note that the examples shown in FIGS. 5(a) to 5(c) are just examples, and the information that can be displayed superimposed on the image is not limited to this. For example, the recording resolution of the image may be displayed. good.

In addition to the full display as shown in Figure 5(a), the on-screen display includes a display as shown in Figure 5(b) that displays only the recording settings, and a display as shown in Figure 5(c) that displays only the recording operation status. There is also a display like this, and whether or not to display each piece of information (display/non-display) may be individually set. The on-screen display mode determined in step S215 refers to a state in which at least one piece of information is set to be displayed in a superimposed manner on the image 500. If the microcomputer 108 determines that the imaging device 100 is in the on-screen display mode (YES in step S215), the process advances to step S216. On the other hand, if the microcomputer 108 determines that the imaging device 100 is not in the on-screen display mode (NO in step S215), the process advances to step S220.

In step S216, the microcomputer 108 refers to the mode information held in the RAM 111 to determine whether the imaging apparatus 100 is in recording operation and the recording command indicating the recording operation state of the imaging apparatus 100 is in the recording state. judge. Recording commands indicating the recording operation state of the imaging apparatus 100 include a recording state (REC) and a recording standby state (STBY, STANDBY). If the microcomputer 108 determines that the recording command is in the recording state (REC) (YES in step S216), the process advances to step S220. If the microcomputer 108 determines that the recording command is not in the recording state (REC), that is, in the recording standby state (STBY, STANDBY) (NO in step S216), the process advances to step S218.

Here, the recording command switches between a recording state and a recording standby state in conjunction with recording start processing and recording end processing of a captured image that are executed in response to a recording control operation using the operation switch group 109. The recording command changes from the recording standby state to the recording state by executing the captured image recording start process, and changes from the recording state to the recording standby state by executing the captured image recording end process. By outputting this recording command together with the output signal of the imaging device 100, an external receiving device connected to the imaging device 100 recognizes that an operation to start recording and an operation to end recording have been performed in the imaging device 100. be able to. An external receiving device connected to the imaging device 100 records input image data using an HDMI signal in accordance with a recording command output from the imaging device 100. That is, the external receiving device connected to the imaging device 100 starts recording input image data when the recording command output from the imaging device 100 switches from the recording standby state to the recording state. Further, the external receiving device connected to the imaging device 100 finishes recording the input image data when the recording command output from the imaging device 100 switches from the recording state to the recording standby state.

In step S217, the microcomputer 108 refers to the mode information held in the RAM 111 and determines whether the imaging device 100 is in the on-screen display mode. If the microcomputer 108 determines that the imaging device 100 is in the on-screen display mode (YES in step S217), the process advances to step S218. If the microcomputer 108 determines that the imaging device 100 is not in the on-screen display mode (NO in step S217), the process advances to step S219.

In step S218, the microcomputer 108 superimposes on-screen information on the development data that has been subjected to the development data correction process in step S213 and is stored in the RAM 111. The on-screen information is information that is set to be displayed in a superimposed manner on an image, among information related to photographing, settings, operating conditions, etc. of an imaging device that can be displayed in a superimposed manner on an image. The on-screen information includes icons, menu screens, character strings (text format), and the like.

In step S219, the microcomputer 108 controls the display resizing unit 104 to resize the developed data and write it to the memory area for HDMI output (VRAM for HDMI output) in the RAM 111. The developed data resized by the display resizing unit 104 in step S219 is the developed data after the development correction generated in step S213, or the developed data on which the on-screen information generated in step S218 is superimposed. Furthermore, the data written to the RAM 111 is data in the YCC4:2:2 format, which will be explained with reference to FIG. After the process in step S219, the process proceeds to step S220.

Here, the memory area for HDMI output (VRAM for HDMI output) to which data is written in steps S210 and S219 will be described with reference to FIGS. 6 to 8.

FIG. 6 is a diagram for explaining a memory area for HDMI output (VRAM for HDMI output) when outputting 12-bit YCC4:2:2 format image data of 4096x2160 pixels by an HDMI signal. As shown in FIG. 6, this memory area 600 has an image area of 4096 pixels in the horizontal direction as shown at 602 and 2160 pixels in the vertical direction as shown in 603. In FIG. 6, as shown by 601, the image data is composed of 2 pixel units (one each of Cb and Cr data for two Y data), and the size of each data is as follows. Each data of Y, Cb, and Cr is 12 bits. The data size of one line (horizontal direction) in this memory area for HDMI output is 4096 (horizontal) x 2 (2 pixels = 4 data (2 Y, 1 Cb, 1 Cr). ) x 12 (bit depth) ÷ 8 (bits to bytes) = 12288 bytes. In addition, the data size for the entire memory area for HDMI output is 4096 (horizontal) x 2160 (vertical) x 2 (4 data for 2 pixels) x 12 (bit depth) ÷ 8 (bit to byte) = 26542080 bytes. be.

FIG. 7 is a diagram for explaining a state in which RAW data is stored in the HDMI output memory area (HDMI output VRAM) explained in FIG. 6 in step S210. Note that the size of the RAW data is 4120 (horizontal) x 2176 (vertical) x 12 (bit depth) ÷ 8 (bit to byte) = 13447680 bytes as described with reference to FIG.

In FIG. 7, as shown in 701, Bayer data (R, Gr data in the first column, In the second column, the Gr and B data) are arranged without any gaps. The size of RAW data is 13447680 bytes, and the data size of one line in the memory area for HDMI output is 12288 bytes. Therefore, as shown at 702 in FIG. 7, 1095 lines (13447680÷12288=1094.375) in the memory area for HDMI output can store RAW data of 4120x2176 pixels and a bit depth of 12 bits. The area after the RAW data (lines 1096 to 2160 in the memory area for HDMI output) is an empty area, so for example, as shown in 703, it corresponds to the RAW data placed up to the 1095th line in this area. You may also place metadata that For example, γ data of RAW data (γ in step S209) may be arranged. Also, for example, parameters used for development processing and correction processing in the imaging device 100 (parameters for RAW data correction processing in step S211, parameters for development processing in step S212, parameters for development data correction processing in step S213). may be placed.

In FIG. 7, a case has been described in which the RAW data written in step S210 fits into the memory area for HDMI output (VRAM for HDMI output). On the other hand, a case in which the RAW data written in step S210 does not fit into the memory area for HDMI output will be described with reference to FIG. 8.

In FIG. 8, 801 is RAW data with 8224x4336 pixels and a bit depth of 10 bits. As explained with reference to FIG. 4, the interior is composed of a Bayer array of R, Gr, Gb, and B. The data size of this RAW data 801 is 8224 (horizontal) x 4336 (vertical) x 10 (bit depth) ÷ 8 (bit to byte) = 44574080 bytes. Therefore, this RAW data 801 does not fit into a memory area (26542080 bytes) corresponding to 4096x2160 pixels and a bit depth of 12 bits. Therefore, the RAW data 801 is divided into two parts, upper and lower, and the RAW data is stored in two frames of memory area for HDMI output. For example, in FIG. 8, the RAW data up to 2176 lines of the RAW data 801 are arranged in the first frame as shown in 802, and the RAW data of the remaining lines of the RAW data 801 are arranged in the second frame as shown in 803. do.

In FIG. 8, 804 is a memory area (first frame) for HDMI output in which RAW data is arranged. The data size of the upper half (2176 lines) of RAW data 801 with 8224x4336 pixels and 10-bit bit depth is 8224 (horizontal) x 2176 (vertical) x 10 (bit depth) ÷ 8 (bit to byte) = 22369280 bytes. . Therefore, the upper half of the RAW data 801 can be stored in 1821 lines (22369280÷12288=1820.41) in a memory area corresponding to 4096x2160 pixels and 12-bit bit depth, as indicated by 805. Since lines 1822 to 2160 in the memory area 804 for HDMI output are empty areas, metadata corresponding to RAW data may be placed, for example, as shown in 806.

807 is a memory area (second frame) for HDMI output in which RAW data is arranged. The data size of the lower half (2160 lines) of RAW data 801 with 8224x4336 pixels and 10-bit bit depth is 8224 (horizontal) x 2160 (vertical) x 10 (bit depth) ÷ 8 (bit to byte) = 22204800 bytes. . Therefore, the lower half of the RAW data 801 can be stored in 1808 lines (22204800÷12288=1807.031) in a memory area corresponding to 4096x2160 pixels and 12-bit bit depth, as shown by 808. Since lines 1809 to 2160 in the memory area 807 for HDMI output are empty areas, metadata corresponding to RAW data may be placed, for example, as shown in 809. In this way, when the RAW data does not fit in the memory area for HDMI output, the RAW data is divided into upper and lower parts, and the RAW data is placed in the memory area for HDMI output (VRAM for HDMI output).

Returning to FIG. 2, in step S220, the microcomputer 108 controls the display resizing unit 104 to resize the developed data and write it to the display output memory area (display output VRAM) in the RAM 111. The developed data resized by the display resizing unit 104 in step S220 is the developed data after the development correction generated in step S213, or the developed data on which the on-screen information generated in step S218 is superimposed. Furthermore, the data written to the RAM 111 is in the YCC4:2:2 format described with reference to FIG.

In step S221, the microcomputer 108 outputs the display output data written to the display output memory area (display output VRAM) in the RAM 111 in step S220 to the display unit 107. Note that since the display unit 107 is compatible with displaying image data in the YCC4:2:2 format, an image corresponding to data for display output is displayed.

In step S222, the microcomputer 108 controls the external output unit 115 to output data for HDMI output to a memory area for HDMI output (VRAM for HDMI output) in the RAM 111 to an external receiving device using an HDMI signal. . Here, the signal output in step S222 is as follows depending on the mode and recording operation state of the imaging device 100.

(1) When the imaging device 100 is in HDMI RAW mode (set to output RAW data) (1-1) If the recording command indicating the recording operation state is in the recording state (REC), or when the imaging device 100 If it is not the on-screen display mode, RAW data has been written to the memory area for HDMI output in step S210. Therefore, a signal in which RAW data is mapped (or stored) to an image area in the YCC4:2:2 format (12 bits) video format is output. Here, the video format of YCC4:2:2 format (12 bits) is a video format defined by the HDMI standard. Note that the video format may also be referred to as a transmission format or an output format.
(1-2) If the imaging device 100 is in the on-screen display mode and the recording command indicating the recording operation state is in the recording standby state (STBY, STANDBY), the memory area for HDMI output is resized in step S219. The developed development data has been written out. On-screen information is superimposed on this development data in step S218. Therefore, an image signal in YCC4:2:2 format on which on-screen information is superimposed is output.

(2) When the imaging device 100 is not in HDMI RAW mode (not set to output RAW data) The developed data resized in step S219 has been written to the memory area for HDMI output. This development data is development data that has been subjected to development data correction processing in step S213, or development data that has been superimposed with on-screen information in step S218. Therefore, an image signal in YCC4:2:2 format on which on-screen information is superimposed according to the settings is output.

Thereafter, the microcomputer 108 repeatedly executes the processes from step S201 onward until it determines in step S204 that there is no HDMI connection or that it determines that the camera mode is not in step S206.

As described above, according to the first embodiment, the imaging apparatus 100 is in the HDMI RAW mode, and when the recording command indicating the recording operation state is in the recording state, the imaging apparatus 100 outputs RAW data to the outside using the HDMI signal. In addition, when the imaging device 100 is in HDMI RAW mode and the recording command is in a recording standby state, the imaging device 100 uses the HDMI signal to generate YCC4:2:2 format image data on which display information such as icons and menu screens can be superimposed. output to the outside. Furthermore, although the imaging device 100 is in the HDMI RAW mode, when it is not in the on-screen display mode, it outputs RAW data to the outside using an HDMI signal.

In this way, even if the imaging device 100 is in the HDMI RAW mode, when it is in the recording standby state, it can control the display of icons, menu screens, etc. by controlling it to output image data in the YCC4:2:2 format. Information can be output. Therefore, even if the imaging device 100 is in HDMI RAW mode, the receiving side can check the information by displaying icons, menu screens, etc., and can check the status of the imaging device 100 when necessary. can.

Note that in step S222 of FIG. 2, if the HDMI RAW mode is in the recording standby state, data is output in a YCC format different from the set mode, so a notification is sent that the output format will be switched by the recording command. You can do it like this. FIGS. 9(a) and 9(b) are diagrams showing examples of displays that notify that the output to the HDMI transmission path has been changed to the YCC format when the HDMI RAW mode is set. For example, as shown in 901 in FIG. 9A, a text message may be used to notify that the output to the HDMI transmission path is switched by a recording start operation (REC operation) that switches the recording command. For example, in FIG. 9(a), the output format may be displayed with an icon by referring to the current state as shown in 902, or the HDMI transmission according to the recording operation state may be displayed as shown in 903. The output format for the route may be displayed on the screen.

Note that in the embodiment described above, the output format to the HDMI transmission path is controlled by the recording command indicating the recording operation state, but the output format may be switched using on-screen display modes that display different amounts of information. For example, among the on-screen displays shown in Figure 5, for the display showing only the recording status in Figure 5(c), information can be notified by the recording command, so regardless of the recording command, the RAW data is stored in the memory area for HDMI output. You can also write and output the data.
Furthermore, in the above description, the development data is image data in YCC4:2:2 format, but is not limited to this, and may be image data in other YCC formats.

<Embodiment 2>
Some imaging devices have an auxiliary function (assist function) that is effective for checking captured image data. When the imaging device is set to HDMI RAW mode and outputs RAW data to the receiving device via the HDMI transmission path, some auxiliary functions (assist functions) cannot be used. Therefore, when the RAW data is output, the receiving device that receives the RAW data cannot confirm the image to which the auxiliary function (assist function) is applied. Therefore, in Embodiment 2, in HDMI RAW mode, if an operation to enable an unusable function is performed on the imaging device, if the HDMI-connected receiving device has the same function, the receiving device will enable the function. Enable the feature. By controlling in this way, even in HDMI RAW mode, it is possible to check an image on the receiving device to which functions equivalent to those that cannot be used on the imaging device are applied. In the following, descriptions of points similar to the first embodiment described above will be omitted, and only points different from the first embodiment described above will be described.

FIG. 10 is a block diagram showing a configuration example of receiving device 1000. The receiving device 1000 is HDMI-connected to the imaging device 100, receives image data via an HDMI transmission path, and displays the received image data.

The receiving unit 1001 receives a signal (HDMI signal) compliant with the HDMI standard. Image data received by the receiving unit 1001 is input to the receiving device 1000. Image data received by the receiving unit 1001 is stored in the RAM 1008. The image processing unit 1002 performs various image processing on the image data received by the receiving unit 1001. The image processing unit 1002 stores the image data after image processing in the RAM 1008.

The display resizing unit 1003 resizes the image data stored in the RAM 1008 to generate display image data in order to output the image data to the display unit 1004 and external output unit 1009. The display resizing unit 1003 stores the generated display image data in the RAM 1008. The display unit 1004 is a display member for displaying display image data stored in the RAM 1008. The display unit 1004 is, for example, a liquid crystal panel.

A microcomputer (microcomputer) 1005 controls the entire receiving device 1000. The operation switch group 1006 is an operation member for the user to input operations. The ROM 1007 is, for example, a flash ROM, and stores programs executed by the microcomputer 1005. Further, a part of the ROM 1007 is used for backup and to hold the state of the system. The RAM 1008 is a volatile memory used by the microcomputer 1005, the image processing unit 1002, and the like as a work memory.

An external output unit 1009 outputs display image data etc. outputted to the RAM 1008 by the image processing unit 1002 to the outside. The external output unit 1009 is, for example, an interface that complies with the HDMI standard or the SDI standard. The external output unit 1009 can output image data for display using signals of standards such as 4K60P and 2K60P. The bus 1010 is a bus that exchanges data between each part of the receiving device 1000.

FIG. 11 is a flowchart showing the processing of the imaging device 100 when an operation to enable a function of the imaging device 100 is performed on the imaging device 100 when the imaging device 100 is in the HDMI RAW mode. Each process in the flowchart of FIG. 11 is controlled by the microcomputer 108 of the imaging device 100 executing a program stored in the ROM 110.

In step S1101, the microcomputer 108 determines whether a menu operation using the operation switch group 109 has been performed. This menu operation is an operation for setting the operation of the imaging device 100, such as setting the resolution of the video signal captured by the image sensor 102 or setting the bit rate for encoding by the compression/expansion unit 114. It is. Furthermore, menu operations include operations for enabling various auxiliary functions (assist functions) that the imaging device 100 has. If the microcomputer 108 determines that a menu operation has been performed (YES in step S1101), the process proceeds to step S1102; otherwise (NO in step S1101), the process waits in step S1101.

Here, the auxiliary functions (assist functions) that the imaging device 100 has include, for example, functions such as Magnification, Anamorphic, False Color, Zebra, Waveform Monitor (WFM), and Marker. Magnification is a function that magnifies and displays a certain area within an image. For example, it is used to check the detailed focus position of a person's eyes when there are display limitations such as a small panel. . Anamorphic is a function that adjusts the displayed image by correcting the image displayed on a display unit such as a liquid crystal panel when photographing using an anamorphic lens that can compress and photograph a horizontally long image. False color is a function that superimposes and displays a pseudo color corresponding to the brightness level on each pixel of an image, allowing you to check the exposure of the image being photographed. Zebra is a function that superimposes diagonal lines according to the brightness level of the image being shot, and notifies areas of overexposed highlights due to overexposure. It can be grasped by The waveform monitor (WFM) is a function that plots the luminance value of an image on the Y axis and the horizontal position of the image on the X axis, and displays the distribution of luminance in the image in a waveform. A marker is a function that displays a mark such as a cross, a frame, or horizontal and vertical lines on the image being photographed. Markers can be used as a guide for composition during shooting, for example, as a guide for the area where you want to capture the subject during shooting, or for editing after shooting by displaying a marker at the position where a subtitle will be placed so that the subject does not overlap. used.

In step S1102, the microcomputer 108 determines whether the menu operated in step S1101 is a function that is exclusive (unusable) in HDMI RAW mode. In the imaging device 100 according to the present embodiment, for example, the above-described functions of Magnification, Anamorphic, False Color, Zebra, Waveform Monitor (WFM), and Marker are functions that are exclusive (unusable) in HDMI RAW mode. If the microcomputer 108 determines that the operated menu is a function that is exclusive (unusable) in HDMI RAW mode (YES in step S1102), the process advances to step S1103. On the other hand, if the microcomputer 108 determines that the operated menu is a function that is not exclusive (usable) in HDMI RAW mode (NO in step S1102), the process advances to step S1109.

In step S1103, the microcomputer 108 controls the external output unit 115 to check whether the hot plug detection signal (HPD) according to the HDMI standard is detected and whether the signal line of the TMDS signal according to the HDMI standard is pulled up. Determine. If the microcomputer 108 determines that the hot plug detection signal is detected and the signal line of the TMDS signal is pulled up (YES in step S1103), the microcomputer 108 determines that the HDMI connection is in progress, and proceeds to step S1104. move on. In other cases (NO in step S1103), the microcomputer 108 determines that there is no HDMI connection, and ends the process of FIG. 11.

In step S1104, the microcomputer 108 controls the external output unit 115 to acquire the EDID of the receiving device 1000 connected to the external output unit 115 via HDMI. EDID is information about a sink device connected via HDMI, and is made up of data such as video formats and functions supported by the sink device, proprietary data of the vendor, and the like. The microcomputer 108 stores the acquired EDID of the receiving device 1000 in the RAM 111.

In step S1105, the microcomputer 108 refers to the EDID of the receiving device 1000 acquired in step S1104 and stored in the RAM 111, and determines whether the receiving device 1000 has a function corresponding to the function operated in step S1101. judge. For example, the imaging device 100 stores in advance a function correspondence table between the imaging device 100 and the receiving device 1000 as shown in Table 1, and based on the EDID acquired from the receiving device 1000, the currently operated function is Search for functions that the corresponding receiving device 1000 has. If the microcomputer 108 determines that the receiving device 1000 has the corresponding function (YES in step S1105), the process advances to step S1106. If the microcomputer 108 determines that the receiving device 1000 does not have the corresponding function (NO in step S1105), the microcomputer 108 ends the process of FIG. 11.

In step S1106, the microcomputer 108 determines whether the image data being output to the receiving device 1000 via the HDMI transmission path is RAW data. If the microcomputer 108 determines that the image data is output as RAW data (YES in step S1106), the process advances to step S1107. If the microcomputer 108 determines that the image data is not output as RAW data (NO in step S1106), the process advances to step S1109.

In step S1107, the microcomputer 108 transmits a control signal to the receiving device 1000 to enable the function of the receiving device 1000 that corresponds to the function operated in step S1101. At this time, the signal to be transmitted may be transmitted in a CEC command format that can be interpreted by the receiving device 1000, or may be written as a signal that can be interpreted by the receiving device 1000 in a metadata portion as shown by 703 in FIG. For example, it is possible to have a table of corresponding functions between the imaging device 100 and the receiving device 1000 as shown in Table 1, and write an ID for specifying the function and information for controlling validity/invalidity etc. in the metadata. Send. For example, when enabling the waveform monitor function shown in Table 1, the imaging device 100 writes “Moni2” and “1” (valid) data in the metadata portion and transmits the data to the receiving device 1000.

In step S1108, the microcomputer 108 receives from the receiving device 1000 the control result based on the control signal transmitted to the receiving device 1000 in step S1107. FIGS. 12(a) to 12(c) are diagrams showing display examples of control results received from the receiving device 1000. For example, the imaging device 100 may display in text the result of controlling the corresponding function in the receiving device 1000 by transmitting the control signal, as shown at 1201 in FIG. 12(a). Further, the imaging device 100 transmits the control signal without checking whether the receiving device 1000 has the corresponding function, and controls the corresponding function in the receiving device 1000, as shown at 1202 in FIG. 12(b). The result and the reason may be displayed in text at the same time. In the example shown in FIG. 12(b), it is displayed that the corresponding function could not be enabled because the receiving device 1000 is a non-compatible monitor that does not have the corresponding function. Further, the display format of the control results is not limited to text display, and may be displayed using icons, for example, as shown at 1203 in FIG. 12(c). Further, in FIG. 12C, as shown at 1204, control results of functions including other output destinations may be displayed in list form on the screen.

In step S1109, the microcomputer 108 performs control to enable the function of the imaging apparatus 100 that was operated in step S1101.

When the imaging device 100 executes the processing described above, functions that cannot be used when outputting RAW data via the HDMI transmission path can be enabled on the HDMI-connected receiving device 1000 side, allowing more settings to be made. This can be done only by operating the imaging device 100.

Next, the operation of the receiving device 1000 connected to the imaging device 100 via HDMI will be described. FIG. 13 is a flowchart showing the processing of the receiving device 1000 connected via HDMI to the imaging device 100 that executes the processing shown in FIG. Each process in the flowchart of FIG. 13 is controlled by the microcomputer 1005 of the receiving device 1000 executing a program stored in the ROM 107.

In step S1301, the microcomputer 1005 controls the receiving unit 1001 to receive the HDMI signal output from the imaging apparatus 100. Here, the received HDMI signal is stored in RAM 1008. Note that the signal received in step S1301 may be a YCC format image data signal or a RAW data signal, but the processing described below is the processing when a RAW data signal is received. It is.

In step S1302, the microcomputer 1005 refers to the HDMI signal received in step S1301 and stored in the RAM 1008, and determines whether the received HDMI signal includes a control signal for a function of the receiving device 1000. Here, the control signal may be transmitted as a CEC command superimposed on the HDMI signal, or may be written in the metadata portion of the HDMI signal and transmitted. If the microcomputer 1005 determines that the received HDMI signal includes a control signal for a function of the receiving device 1000 (YES in step S1302), the process advances to step S1203. If the microcomputer 1005 determines that the received HDMI signal does not include a control signal for a function of the receiving device 1000 (NO in step S1302), the process in FIG. 13 ends.

In step S1303, the microcomputer 1005 controls the function specified by the HDMI signal received in step S1301. For example, if the signals “Moni2” and “1” in the example shown in Table 1 are written in the metadata part of the received HDMI signal, the microcomputer 1005 enables the waveform monitor function of the receiving device 1000. control like this.

In step S1304, the microcomputer 1005 transmits to the imaging apparatus 100 the control result of the function based on the control signal included in the HDMI signal received in step S1301.

According to the second embodiment, a function that cannot be enabled on the imaging device 100 side during HDMI RAW mode can be enabled on the receiving device 1000 side, and an image to which the function is applied can be checked on the receiving device. It becomes possible to do so. For example, it is possible to check on the receiving device 100 auxiliary functions (assist functions) such as a waveform monitor and markers that are effective in checking captured image data.

Note that in this embodiment, the control result of the function based on the control signal is transmitted from the receiving device 1000 to the imaging device 100 and displayed on the imaging device 100, but the process of displaying this control result is not essential; Optional.

Note that the present invention is not limited to these specific embodiments, and includes various forms without departing from the gist of the present invention. Furthermore, each of the embodiments described above merely shows one embodiment of the present invention, and it is also possible to combine the embodiments as appropriate.

(Other embodiments of the present invention)
The present invention provides a system or device with a program that implements one or more functions of the embodiments described above via a network or a storage medium, and one or more processors in a computer of the system or device reads and executes the program. This can also be achieved by processing. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

Note that the above-mentioned embodiments are merely examples of embodiments of the present invention, and the technical scope of the present invention should not be construed as limited by these embodiments. That is, the present invention can be implemented in various forms without departing from its technical idea or main features.

The disclosure of this embodiment includes the following configuration, method, etc.
(Configuration 1)
Output means for outputting RAW data or developed data generated by developing the RAW data to the outside;
When the setting is to output the RAW data, if the operating state regarding image recording is a recording state, the RAW data is output by the output means, and if not the recording state, the developed data is output by the output means. a control means for controlling the
An imaging device comprising:
(Configuration 2)
The control means controls such that when outputting the developed data, display information is superimposed on the developed data and output, and when outputting the RAW data, the display information is output without being superimposed. The imaging device according to configuration 1, characterized in that:
(Configuration 3)
According to configuration 1 or 2, the control means controls the output means to output the RAW data and a recording command indicating that the recording state is in the recording state. imaging device.
(Configuration 4)
In the configuration 2, the display information superimposed on the developed data when the recording state is not set includes information notifying that the output will be switched to the RAW data when the recording state is entered. The imaging device described.
(Configuration 5)
The control means is configured to output the RAW data, and when the display information is not displayed, the control means controls the RAW data to be outputted by the output means. 4. The imaging device according to any one of 4.
(Configuration 6)
6. The imaging device according to any one of configurations 1 to 5, wherein the development data is YCC format image data generated based on the RAW data.
(Configuration 7)
7. The imaging apparatus according to configuration 6, wherein the control unit controls the RAW data to be arranged and output in an image area in the YCC video format.
(Configuration 8)
8. The imaging device according to any one of configurations 1 to 7, wherein the output means outputs the RAW data or the developed data to the outside using a signal compliant with the HDMI standard.
(Configuration 9)
The imaging device according to configuration 2, wherein the display information includes at least one of recording resolution, color information, bit depth, and ISO value.
(Configuration 10)
If an operation to enable a function that cannot be used in the imaging device is performed when outputting the RAW data, the control means enables a function corresponding to the function that cannot be used in a receiving device that receives the RAW data. 10. The imaging device according to any one of configurations 1 to 9, wherein the imaging device is controlled to transmit a control signal that controls the imaging device.
(Configuration 11)
In the configuration 10, the control means determines whether or not the receiving device has a function corresponding to the unavailable function, and performs control to transmit the control signal according to the result of the determination. The imaging device described.
(Configuration 12)
12. The imaging device according to configuration 10 or 11, wherein the control signal is transmitted as metadata of a signal compliant with the HDMI standard.
(Configuration 13)
12. The imaging device according to configuration 10 or 11, wherein the control signal is transmitted as a CEC command in a signal compliant with the HDMI standard.
(Configuration 14)
14. The imaging device according to any one of configurations 10 to 13, characterized in that a result of controlling a function in the receiving device using the control signal is displayed.
(Method 1)
an output step of outputting RAW data or developed data generated by developing the RAW data to the outside by an output means;
When the setting is to output the RAW data, if the operating state regarding image recording is a recording state, the RAW data is output by the output means, and if not the recording state, the developed data is output by the output means. a control process for controlling the
A method for controlling an imaging device, comprising:
(Program 1)
In the computer of the imaging device,
an output step of outputting RAW data or developed data generated by developing the RAW data to the outside by an output means;
When the setting is to output the RAW data, if the operating state regarding image recording is a recording state, the RAW data is output by the output means, and if not the recording state, the developed data is output by the output means. a control step for controlling the
A program to run.

100: Imaging device 103, 1002: Image processing section 104, 1003: Display resizing section 105: Recording resizing section 106: OSD generation section 107, 1004: Display section 108, 1005: Microcomputer 109, 1006: Operation switch group 110, 1007: ROM 111, 1008: RAM 115, 1009: External output section 1000: Receiving device 1001: Receiving section

Claims (16)

Output means for outputting RAW data or developed data generated by developing the RAW data to the outside;
When the setting is to output the RAW data, if the operating state regarding image recording is a recording state, the RAW data is output by the output means, and if not the recording state, the developed data is output by the output means. a control means for controlling the
An imaging device comprising: The control means controls such that when outputting the developed data, display information is superimposed on the developed data and output, and when outputting the RAW data, the display information is output without being superimposed. The imaging device according to claim 1, characterized in that:. 2. The control means controls the output means to output the RAW data and a recording command indicating the recording state in the recording state. Imaging device. 2. The display information superimposed on the developed data when the recording state is not set includes information notifying that the output of the RAW data will be switched when the recording state is entered. The imaging device described in . 1. The control means is configured to output the RAW data, and when display information is not displayed, the control means controls the RAW data to be outputted by the output means. The imaging device described in . The imaging device according to claim 1, wherein the development data is YCC format image data generated based on the RAW data. 7. The imaging apparatus according to claim 6, wherein the control means controls the RAW data to be arranged and output in an image area in the YCC video format. 7. The imaging apparatus according to claim 6, wherein the output means outputs the RAW data or the developed data to the outside using a signal compliant with the HDMI standard. The imaging device according to claim 2, wherein the display information includes at least one of recording resolution, color information, bit depth, and ISO value. If an operation to enable a function that cannot be used in the imaging device is performed when outputting the RAW data, the control means enables a function corresponding to the function that cannot be used in a receiving device that receives the RAW data. The imaging device according to any one of claims 1 to 5, wherein the imaging device is controlled to transmit a control signal that The control means determines whether or not the receiving device has a function corresponding to the unavailable function, and controls to transmit the control signal according to the result of the determination. 11. The imaging device according to 10. The imaging device according to claim 10, wherein the control signal is transmitted as metadata of a signal compliant with the HDMI standard. The imaging apparatus according to claim 10, wherein the control signal is transmitted as a CEC command in a signal compliant with the HDMI standard. 11. The imaging device according to claim 10, wherein a result of controlling a function in the receiving device using the control signal is displayed. an output step of outputting RAW data or developed data generated by developing the RAW data to the outside by an output means;
When the setting is to output the RAW data, if the operating state regarding image recording is a recording state, the RAW data is output by the output means, and if not the recording state, the developed data is output by the output means. a control process for controlling the
A method for controlling an imaging device, comprising: In the computer of the imaging device,
an output step of outputting RAW data or developed data generated by developing the RAW data to the outside by an output means;
When the setting is to output the RAW data, if the operating state regarding image recording is a recording state, the RAW data is output by the output means, and if not the recording state, the developed data is output by the output means. a control step for controlling the
A program to run.

Images