WO2021246171A1 - 撮像装置、および画像処理方法、並びにプログラム - Google Patents

撮像装置、および画像処理方法、並びにプログラム Download PDF

Info

Publication number
WO2021246171A1
WO2021246171A1 PCT/JP2021/018917 JP2021018917W WO2021246171A1 WO 2021246171 A1 WO2021246171 A1 WO 2021246171A1 JP 2021018917 W JP2021018917 W JP 2021018917W WO 2021246171 A1 WO2021246171 A1 WO 2021246171A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
focus
pixel
unit
change
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2021/018917
Other languages
English (en)
French (fr)
Japanese (ja)
Inventor
遼太 宮澤
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Sony Group Corp
Original Assignee
Sony Group Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Sony Group Corp filed Critical Sony Group Corp
Priority to JP2022528528A priority Critical patent/JP7632463B2/ja
Priority to US17/921,495 priority patent/US12170845B2/en
Publication of WO2021246171A1 publication Critical patent/WO2021246171A1/ja
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/67—Focus control based on electronic image sensor signals
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/67—Focus control based on electronic image sensor signals
    • H04N23/673—Focus control based on electronic image sensor signals based on contrast or high frequency components of image signals, e.g. hill climbing method
    • G—PHYSICS
    • G03—PHOTOGRAPHY; CINEMATOGRAPHY; ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ELECTROGRAPHY; HOLOGRAPHY
    • G03B—APPARATUS OR ARRANGEMENTS FOR TAKING PHOTOGRAPHS OR FOR PROJECTING OR VIEWING THEM; APPARATUS OR ARRANGEMENTS EMPLOYING ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ACCESSORIES THEREFOR
    • G03B17/00—Details of cameras or camera bodies; Accessories therefor
    • G03B17/18—Signals indicating condition of a camera member or suitability of light
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/14—Digital output to display device ; Cooperation and interconnection of the display device with other functional units
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/20—Image enhancement or restoration using local operators
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
    • G06V10/75—Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries
    • G06V10/751—Comparing pixel values or logical combinations thereof, or feature values having positional relevance, e.g. template matching
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G5/00—Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators
    • G09G5/36—Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators characterised by the display of a graphic pattern, e.g. using an all-points-addressable [APA] memory
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/63—Control of cameras or camera modules by using electronic viewfinders
    • H04N23/631—Graphical user interfaces [GUI] specially adapted for controlling image capture or setting capture parameters
    • H04N23/632—Graphical user interfaces [GUI] specially adapted for controlling image capture or setting capture parameters for displaying or modifying preview images prior to image capturing, e.g. variety of image resolutions or capturing parameters
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/63—Control of cameras or camera modules by using electronic viewfinders
    • H04N23/633—Control of cameras or camera modules by using electronic viewfinders for displaying additional information relating to control or operation of the camera
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/63—Control of cameras or camera modules by using electronic viewfinders
    • H04N23/633—Control of cameras or camera modules by using electronic viewfinders for displaying additional information relating to control or operation of the camera
    • H04N23/635—Region indicators; Field of view indicators
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10024—Color image
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G2354/00—Aspects of interface with display user

Definitions

  • the present disclosure relates to an image pickup apparatus, an image processing method, and a program. More specifically, the present invention relates to an image pickup apparatus that generates and displays information that enables confirmation of the in-focus state when the focus adjustment process is executed, an image processing method, and a program.
  • High-performance, high-resolution image pickup devices are capable of manual focus adjustment.
  • the cameraman who is the user, looks at the image (through image) of the viewfinder or the monitor, confirms the change in the resolution level of the image, and determines whether or not the optimum focus adjustment has been made for the target subject.
  • the display unit such as a view finder or monitor often has a smaller number of pixels than the image sensor (image sensor), and the display unit such as the view finder or monitor sufficiently expresses the change in resolution. This is because it cannot output. As a result, it becomes difficult for the user (cameraman) to identify a subtle difference in focus level.
  • Patent Document 1 International Publication WO2016 / 163324
  • Patent Document 2 Japanese Patent Laid-Open No. 2009-272784
  • Patent Document 1 International Publication No. WO2016 / 163324 provides a peaking signal generated by an HPF or the like in an output image of an image sensor (image sensor), that is, a peaking signal for identifying a region having a high degree of focus in the image.
  • image sensor image sensor
  • High-frequency signals increase in the region with high in-focus degree, and by using a filtered image by HPF, it is possible to select the region with high in-focus degree, and peaking such as a red signal in the region with high in-focus degree. By superimposing and displaying the signals, the user can determine the in-focus area.
  • Patent Document 2 Japanese Unexamined Patent Publication No. 2009-272784
  • the in-focus degree level is determined according to the ratio of high-frequency components contained in the through image, and the determined result is indicated by the length of the bar.
  • the configuration is disclosed in which the in-focus level display unit of the mold is displayed together with the through image.
  • the peaking signal disclosed in Patent Document 1 is not limited to the object to be focused, and is superimposed and displayed on the entire through image, and further superimposed even after the focus adjustment is completed, so that the imaged object becomes difficult to see. There is.
  • the present disclosure has been made in view of the above problems, for example, and includes an image pickup device that generates and displays information that enables confirmation of the in-focus state in a predetermined pixel area unit, an image processing method, and a program.
  • the purpose is to provide.
  • the first aspect of this disclosure is An in-focus degree change analysis unit that analyzes a change in the in-focus degree between the current image output from the image sensor and the past image output from the image sensor before the output of the current image.
  • a display control unit that uses the analysis result of the in-focus degree change analysis unit to generate display data that enables confirmation of the in-focus state from the past image in units of pixel areas of the current image.
  • the image pickup apparatus has a display unit for displaying the display data.
  • the second aspect of the present disclosure is It is an image processing method executed in an image processing device.
  • the in-focus degree change analysis unit analyzes the change in the in-focus degree between the current image output from the image sensor and the past image output from the image sensor before the output of the current image.
  • the display unit is in an image processing method having a display step for displaying the display data.
  • the third aspect of the present disclosure is A program that executes image processing in an image processing device.
  • Focus change analysis step for causing the in-focus degree change analysis unit to analyze the change in the in-focus degree between the current image output from the image sensor and the past image output from the image sensor before the output of the current image.
  • a display control step for causing the display control unit to generate display data capable of confirming the in-focus state from the past image in units of the pixel area of the current image by using the analysis result of the in-focus degree change analysis unit.
  • the program of the present disclosure is, for example, a program that can be provided by a storage medium or a communication medium provided in a computer-readable format to an information processing device or a computer system capable of executing various program codes.
  • a program can be provided by a storage medium or a communication medium provided in a computer-readable format to an information processing device or a computer system capable of executing various program codes.
  • system is a logical set configuration of a plurality of devices, and the devices of each configuration are not limited to those in the same housing.
  • a device and a method for analyzing a change in focus between a current image and a past image and outputting display data capable of identifying the change in focus to a display unit are realized.
  • an in-focus degree change analysis unit that analyzes a change in the in-focus degree between the current image output from the image sensor and the past image output from the image sensor before the output of the current image is used. It has a display control unit that uses the analysis result of the focus change analysis unit to generate display data that enables confirmation of the focus change state from the past image in units of pixel areas of the current image.
  • each pixel is either (a) a pixel that is approaching focus, (b) a pixel that is out of focus, or (c) a pixel that does not change the degree of focus.
  • Generate and display classification information With this configuration, a device and a method for analyzing changes in the in-focus degree between the current image and the past image and outputting display data capable of identifying the change in the in-focus degree to the display unit are realized. It should be noted that the effects described in the present specification are merely exemplary and not limited, and may have additional effects.
  • FIG. 1 It is a figure explaining the configuration example of the image pickup apparatus of this disclosure. It is a figure explaining the structural example of the focusing degree change analysis part. It is a figure explaining the pixel classification information. It is a figure explaining the example of display data. It is a figure explaining the example of display data. It is a figure explaining the example of display data. It is a figure explaining the example of display data. It is a figure explaining the specific example of the pixel analysis processing executed by the pixel classification part of the focusing degree change analysis part. It is a figure explaining the specific example of the pixel analysis processing executed by the pixel classification part of the focusing degree change analysis part. It is a figure explaining the structural example of the focusing degree change analysis part of Example 2.
  • FIG. 1 It is a figure explaining the configuration example of the image pickup apparatus of this disclosure. It is a figure explaining the structural example of the focusing degree change analysis part. It is a figure explaining the pixel classification information. It is a figure explaining the example of display data. It is a figure explaining the example
  • FIG. 1 It is a figure explaining the specific example of the characteristic of HPF, and the example of the focus degree change analysis. It is a figure explaining the specific example of the characteristic of HPF, and the example of the focus degree change analysis. It is a figure explaining the structural example of the focusing degree change analysis part of Example 3.
  • FIG. 1 It is a figure explaining the specific example of the characteristic of HPF, and the example of the focus degree change analysis. It is a figure explaining the specific example of the characteristic of HPF, and the example of the focus degree change analysis. It is a figure explaining the structural example of the focusing degree change analysis part of Example 3. FIG.
  • FIG. 1 is a block diagram showing a main configuration of the image pickup apparatus 100 of the present disclosure.
  • the image pickup device 100 includes an input unit 101, a control unit 102, a drive unit 103, an image sensor (image sensor) 104, an in-focus degree change analysis unit 105, a display control unit 106, and a display unit 107. ..
  • FIG. 1 is a diagram showing only the main configuration used for the processing of the present disclosure in the configuration of the image pickup apparatus 100.
  • the image pickup apparatus 100 has various components such as a lens and a storage unit in addition to the configuration shown in this figure.
  • the input unit 101 corresponds to a user's operation unit, and is an input unit for performing, for example, a shutter, focus adjustment, and various other operations and settings.
  • the control unit 102 controls various processes executed by the image pickup apparatus 100.
  • the control unit 102 includes, for example, a processor that executes processing according to a program stored in a storage unit (not shown).
  • the drive unit 103 executes, for example, a drive process based on a user operation on the input unit 101. For example, a lens drive process for adjusting the focus is executed.
  • the image sensor (image sensor) 104 is an image sensor for generating a subject image input through a lens (not shown). For example, it is composed of CMOS, CCD and the like.
  • the output of the image sensor (image sensor) 104 is the image sensor (image sensor) output image 121 shown in the figure.
  • the image sensor (image sensor) output image 121 is input to the in-focus degree change analysis unit 105.
  • the in-focus degree change analysis unit 105 analyzes the temporal change in the in-focus degree in units of, for example, pixels or a predetermined number of pixel areas.
  • the details of the configuration and processing of the in-focus degree change analysis unit 105 will be described later.
  • the current image sensor output image is compared with the past image sensor output image one to several frames ago to present.
  • an analysis process for classifying which of the following pixels corresponds to is executed. (1) Pixels approaching focus (2) Pixels out of focus (3) Pixels with no change in focus
  • the details of the configuration and processing of the focus change analysis unit 105 will be described later. explain.
  • the display control unit 106 performs a development process for displaying the image sensor (image sensor) output image 121 on the display unit 107, and a focus change state based on the pixel classification information output from the focus change analysis unit 105. Processing for generating pixel-based focusing degree change information for showing, processing for superimposing the generated pixel-based focusing degree change information on the developed image of the image sensor (image sensor) output image 121, and the like are performed, and the display unit 107 is displayed. Generate a display image to be output.
  • the display unit 107 is a display unit that displays a display image generated by the display control unit 106, and is composed of, for example, a monitor or a view finder. Specifically, it is a display unit such as an LCD.
  • FIG. 2 is a block diagram showing an example (Example 1) of the detailed configuration of the in-focus degree change analysis unit 105, which is a component of the image pickup apparatus 100 shown in FIG.
  • the in-focus degree change analysis unit 105 of the first embodiment has a memory 151, a first HPF (High Pass Filter) 152, a second HPF (High Pass Filter) 153, and a pixel classification unit 154.
  • the image sensor output image 121 which is the output of the image sensor (image sensor) 104, is input to the first HPF 152 of the in-focus degree change analysis unit 105 and is stored in the memory 151.
  • the latest image sensor output image 121 that is, the current image 122 is input to the first HPF 152, and the first HPF 152 performs a filtering process applying the HPF to the current image 122.
  • a pixel region with a high degree of focus has a high frequency
  • a pixel region with a low degree of focus has a low frequency and a large number of low-frequency signals. It is possible to generate an HPF filtering image in which a high frequency region and a low frequency region can be identified.
  • the HPF filtering image is, for example, an image in which a signal (pixel value) corresponding to the degree of focus (high frequency component amount) is set.
  • the first HPF 152 generates a current image HPF filtering image, and inputs the generated current image HPF filtering image to the pixel classification unit 154.
  • the memory 151 is an image frame memory, and is a memory capable of storing one image frame to several image frames.
  • the image sensor output image 121 which is the output of the image sensor (image sensor) 104, is sequentially input to and stored in the memory 151. From the memory 151, the past image 123 one image frame to several image frames before the current image 122 is read out and input to the second HPF 153.
  • a past image 123 one image frame to several image frames before the current image 122 is input to the second HPF 153, and filtering processing is performed by applying the HPF to the past image 123.
  • the second HPF 153 generates an HPF filtering image capable of distinguishing a region having a high degree of focus and a region having a low degree of focusing of the past image 123 by a filtering process to which the HPF is applied, and the generated past image HPF filtering image is sent to the pixel classification unit 154. input.
  • the pixel classification unit 154 inputs the following two HPF filtering images. (1) HPF filtering image for the current image 122 (2) HPF filtering image for the past image 123
  • the pixel classification unit 154 compares these two HPF filtering images and classifies which of the following each pixel of the current image corresponds to. (1) Pixels approaching focus (2) Pixels out of focus (3) Pixels with no change in focus
  • the pixel classification unit 154 compares the high frequency signal component of the corresponding pixel region between the current image HPF filtering image and the past image HPF filtering image. As described above, if there are many high frequency components, it can be determined that the degree of focus is high.
  • the pixel classification unit 154 determines that the pixel is "a pixel approaching focus”. Judge that there is.
  • the pixel is "a pixel out of focus”. do.
  • the pixel is determined to be a "pixel whose focus degree does not change".
  • the pixel classification unit 154 analyzes the change in the high frequency component signal for each pixel corresponding to the current image HPF filtering image and the past image HPF filtering image, and the analysis result shows the change state of the focusing degree.
  • pixel classification information for each pixel that is, which of the following three types of pixel classification each pixel corresponds to is analyzed, and the analysis result is output to the display control unit 106 as pixel classification information 124.
  • the pixel classification unit 154 classifies pixels based on the difference between the high frequency component signal of the current image and the high frequency component signal of the past image. Therefore, even in a pixel region having many high frequency components such as a signal component of the original image, for example, a texture region, if the amount of change in the high frequency component is small according to the focus adjustment, the pixel has no change in focus. It is judged. That is, by applying the process of the present disclosure, it is possible to display the in-focus state confirmation data that does not interfere with the manual focus.
  • FIG. 3 is a diagram illustrating pixel classification information 124 generated by the pixel classification unit 154. As shown in FIG. 3, the pixel classification information 124 is provided. (Category 1) Pixels approaching focus (Category 2) Pixels out of focus (Category 3) Pixels with no change in focus Each pixel is classified into the above three types.
  • Pixels that are approaching focus are pixels whose in-focus degree is higher than the past image in the current image by a threshold value or more.
  • Pixels that are out of focus are pixels whose in-focus degree is lower than the past image in the current image by a threshold value or more.
  • Category 3) A pixel having no change in the degree of focus is a pixel in which the change in the degree of focus is less than the threshold value between the past image and the current image.
  • the pixel classification information 124 generated by the pixel classification unit 154 is output to the display control unit 106.
  • the display control unit 106 is based on the development process for displaying the image sensor (imaging element) output image 121 on the display unit 107 and the pixel classification information output from the in-focus degree change analysis unit 105. It performs processing such as generating pixel-based focusing degree change information to indicate the focusing degree change state, and superimposing the generated pixel-based focusing degree change information on the developed image of the image sensor (imaging element) output image 121. , Generates display data 125 to be output to the display unit 107. The display data 125 generated by the display control unit 106 is displayed on the display unit 107.
  • FIG. 4 shows an example of transition of an image displayed on the display unit 107 in the process of adjusting the focus by the user (cameraman).
  • FIG. 4 (1) is a through image, which is an image of the image sensor (image sensor) output image 121 after development processing.
  • the user (cameraman) starts focus adjustment while looking at this through image.
  • the pixel classification unit 154 classifies each pixel in the image into the following three types. (Category 1) Pixels approaching focus (Category 2) Pixels out of focus (Category 3) Pixels with no change in focus
  • the pixel classification information 124 is input to the display control unit 106, and the display control unit 106 uses this classification information to generate display data 125.
  • the example shown in FIG. 4 is an example of generating display data in which only "(classification 1) pixels approaching focus” are selected and this "(classification 1) pixels approaching focus” can be identified. Is.
  • FIG. 4 (2a) Pixel-by-pixel-by-focus degree change information superimposed image is a change in focus degree for making it possible to identify "(Category 1) pixels approaching focus” in the image of "house” in the image.
  • This is display data in which a discrimination signal (peaking signal), for example, a specific color signal (for example, a red signal) is superimposed.
  • the user By looking at this display data, the user (cameraman) can confirm that the "house” area in the image is approaching focus.
  • FIG. 4 (2b) Pixel-unit focusing degree change information superimposed image is a change in focusing degree for making it possible to identify "(Category 1) pixels approaching focusing" in the image of "people" in the image.
  • This is display data in which a discrimination signal (peaking signal), for example, a specific color signal (for example, a red signal) is superimposed.
  • the user By looking at this display data, the user (cameraman) can confirm that the area of "people” in the image is approaching focus. In this way, it is easy to check the degree of focus only in the "people" area.
  • the example of the display data shown in FIG. 4 is a simple explanatory diagram for explaining the process of the present disclosure in an easy-to-understand manner. In reality, it is unlikely that the focus change determination signal for each house or person will be displayed, and the focus change determination signal (peaking) in finer units such as a part of the house or a part of the person. Signal) is displayed.
  • the example shown in FIG. 4 is an example of display data superposed with an in-focus degree change determination signal capable of identifying only "(classification 1) pixels approaching in-focus", but the display control unit 106 For example, it is also possible to generate display data in which only the “(classification 2) out-of-focus pixels” are superimposed with the in-focus degree change determination signal that can be identified and display them on the display unit 107. ..
  • FIG. 5 is an example of generating display data in which only "(classification 2) out-of-focus pixels" is selected and this "(classification 2) out-of-focus pixels" can be identified. be.
  • FIG. 5 (1) is a through image, which is an image of the image sensor (image sensor) output image 121 after development processing.
  • the user (cameraman) starts focus adjustment while looking at this through image.
  • FIG. 5 (2a) In the pixel-by-pixel unit focus change information superimposed image, the focus degree for making it possible to identify "(classification 2) pixels out of focus” in the image of "tree” in the image.
  • This is display data in which a change determination signal, for example, a specific color signal (for example, a blue signal) is superimposed.
  • the user By looking at this display data, the user (cameraman) can confirm that the "tree" area in the image is out of focus.
  • FIG. 4 (2b) In the pixel-by-pixel unit focusing degree change information superimposed image, the focusing degree for making it possible to identify "(classification 2) pixels out of focus” in the image of "house” in the image.
  • This is display data in which a change determination signal, for example, a specific color signal (for example, a blue signal) is superimposed.
  • the user By looking at this display data, the user (cameraman) can confirm that the "house” area in the image is out of focus.
  • FIG. 4 is an example of image data in which only the pixel area of “(Category 1) pixels approaching focus” can be identified, and FIG. 5 shows “(Category 2) out of focus”.
  • This is an example of image data in which the pixel area of only "pixels” can be identified, but the display control unit 106 can also identify only "(classification 3) pixels having no change in focus degree”. It is also possible to generate data.
  • the display control unit 106 can also generate display data in which all of the following three types of pixels or two types can be identified together. (Category 1) Pixels approaching focus (Category 2) Pixels out of focus (Category 3) Pixels with no change in focus
  • a red focus change determination signal is superimposed on "(Category 1) Pixels approaching focus”.
  • a blue focus change determination signal is superimposed on "(Category 2) Pixels that are out of focus”.
  • the "(classification 3) pixel having no change in focus degree” may be configured to generate display data on which a yellow focus degree change determination signal is superimposed and display it on the display unit 107.
  • the user By looking at the display data displayed on the display unit 107, the user (cameraman) can easily and surely identify the in-focus degree change mode for each pixel region in the image.
  • the display control unit 106 may be configured to generate data displayed by using the bar indicator and output the degree of focus of the pixel region approaching focus to the display unit 107, for example.
  • FIG. 6 shows a specific example.
  • the “human” region in the image is the “(classification 1) pixel approaching focus” region, respectively.
  • a bar indicator indicating the degree of focus is displayed at the lower side of the image.
  • This bar indicator is a level indicator of the degree of focus indicating the level of the degree of focus in the "(classification 1) pixels approaching focus” region by the length of the bar.
  • the length of the bar is longer in (2c), and the user (cameraman) can see the "person” in the "(classification 1) pixel approaching focus” area. It is possible to correctly recognize that the degree of focus of the pixel region is gradually increasing.
  • the in-focus degree change analysis unit 105 of the image pickup apparatus 100 of the present disclosure analyzes the change in the in-focus degree of each pixel, and classifies each pixel into the following three types of pixels based on the analysis result. .. (Category 1) Pixels approaching focus (Category 2) Pixels out of focus (Category 3) Pixels with no change in focus
  • the display control unit 106 uses this classification result to generate display data in which the in-focus degree change mode of each pixel can be recognized and displays it on the display unit 107.
  • the user By performing these processes, the user (cameraman) can see the image displayed on the display unit 107 and accurately and easily identify the change in the in-focus degree of each pixel.
  • the in-focus degree change analysis unit 105 analyzes the temporal change in the in-focus degree in units of, for example, pixels or a predetermined number of pixel areas. For example, the current image sensor output image is compared with the past image sensor output image one to several frames ago, and each pixel of the current image sensor output image is classified as which of the following is applicable. do. (Category 1) Pixels approaching focus (Category 2) Pixels out of focus (Category 3) Pixels with no change in focus
  • the pixel classification unit 154 of the in-focus degree change analysis unit 105 has a current image HPF filtering image, a past image HPF filtering image, and these two HPF filtering images. , And analyze which of the above (Category 1) to (Category 3) each of the pixels of the current image corresponds to.
  • the graph shown in FIG. 7 is a graph showing time on the horizontal axis and focusing degree (HPF result) on the vertical axis.
  • the captured image at time t1 corresponds to the past image, and the captured image at time t2 corresponds to the current image.
  • the past image which is the captured image at time t1
  • the current image which is the captured image at time t2
  • the HPF in the first HPF 152 shown in FIG. 2 is filtered by the HPF in the first HPF 152 shown in FIG. 2, and the amount of high frequency components for each pixel is calculated.
  • the pixel classification unit 154 compares the current image HPF filtering image, the past image HPF filtering image, and these two HPF filtering images, and each pixel of the current image is described in the above (classification 1) to (classification 3). It is analyzed which of the pixels corresponds to.
  • the pixel A shown in FIG. 7 has a lower in-focus degree of the current image than the past image, and the in-focus degree difference (high-frequency component amount difference) is equal to or more than the threshold value. "are categorized. Further, the pixel B has a higher in-focus degree of the current image than the past image, and the in-focus degree difference (high-frequency component amount difference) is equal to or more than the threshold value, so that the pixel B is classified as "a pixel approaching in-focus". .. Further, the pixel C is classified as "a pixel having no change in the in-focus degree” because the difference between the in-focus degree of the past image and the present image (difference in the amount of high-frequency component) is less than the threshold value.
  • the in-focus degree change analysis process in the in-focus degree change analysis unit 105 is repeated for images continuously input from the image sensor (image sensor) 104, and the process is executed. That is, the process of generating the latest classification result for the latest input image is repeatedly executed for each new input image.
  • FIG. 8 is a graph showing a change in the degree of focus of one pixel, “pixel A”, while the user (cameraman) continuously executes the focus adjustment. Similar to FIG. 7, the graph shown in FIG. 8 is a graph showing time on the horizontal axis and focusing degree (HPF result) on the vertical axis.
  • the difference in focus difference in the amount of high frequency components
  • the difference in focus difference in the amount of high frequency components
  • the in-focus degree of the current image (t21 photographed image) is higher than that of the past image (t20 photographed image), and the in-focus degree difference (high frequency component amount difference) is equal to or more than the threshold value. It is classified as "a pixel approaching the focus”.
  • the in-focus degree of the current image (t31 photographed image) is lower than that of the past image (t30 photographed image), and the in-focus degree difference (high-frequency component amount difference) is equal to or more than the threshold value. It is classified as "a pixel that is out of focus”.
  • the difference in the degree of focus (difference in the amount of high frequency components) between the past image (image taken at t40) and the current image (image taken at t41) is less than the threshold value. It is classified as "no pixel”.
  • the in-focus degree change analysis process in the in-focus degree change analysis unit 105 is repeated for the images continuously input from the image sensor (image sensor) 104, and the process is executed.
  • the display control unit 106 sequentially updates the display data according to the latest pixel classification information input from the focus degree change analysis unit 105, generates new display data, and outputs the new display data to the display unit 107.
  • Example 2 About the example of the in-focus degree change analysis unit that performs pixel classification using the averaging data for each of a plurality of pixel areas]
  • Example 2 an example of the in-focus degree change analysis unit that performs pixel classification using averaging data in units of a plurality of pixel areas will be described.
  • the in-focus degree change analysis unit 105 described above with reference to FIG. 2 executes a process of classifying each pixel as a pixel corresponding to any of the following in the pixel classification unit 154 in the pixel classification unit 154. It is a configuration to do. (Category 1) Pixels approaching focus (Category 2) Pixels out of focus (Category 3) Pixels with no change in focus
  • FIG. 9 is a block diagram showing a configuration example of the in-focus degree change analysis unit 105b of the second embodiment. That is, it is a block diagram which shows the structural example of the focusing degree change analysis unit 105b which performs pixel classification using the averaging data of a plurality of pixel area units.
  • the in-focus degree change analysis unit 105b shown in FIG. 9 has a pixel area unit first HPF output averaging unit 201 and a pixel area unit second HPF output in the in-focus degree change analysis unit 105 described above with reference to FIG. This is a configuration in which the averaging unit 202 is added.
  • the pixel area unit first HPF output averaging unit 201 performs averaging processing using the pixel values of surrounding pixels for each pixel value of the current image HPF filtering image output from the first HPF 152 in the previous stage. For example, for one pixel of the HPF filtering image, the averaged pixel value of the pixel values in the 9 ⁇ 9 pixel region around the pixel is calculated and used as the pixel value of the one pixel (HPF filtering result).
  • the pixel area unit second HPF output averaging unit 202 performs averaging processing using the pixel values of the surrounding pixels for each pixel value of the past image HPF filtering image output from the second HPF 153 in the previous stage. For example, for one pixel of the HPF filtering image, the averaged pixel value of the pixel values in the 9 ⁇ 9 pixel region around the pixel is calculated and used as the pixel value of the one pixel (HPF filtering result).
  • the first HPF 152 inputs the latest image sensor output image 121, that is, the current image 122, performs filtering processing by applying the HPF to the current image 122, and performs the filtering process of the current image 122. Generate the corresponding HPF filtered image.
  • the second HPF 153 inputs the past image 123 one image frame to several image frames before the current image 122, performs filtering processing by applying the HPF to the past image 123, and generates an HPF filtering image corresponding to the past image 123. do.
  • a pixel region with a high degree of focus has a high frequency
  • a pixel region with a low degree of focus has a small high frequency and a large number of low frequency signals. It is possible to generate an HPF filtered image in which low regions can be identified.
  • the Nyquist frequency is a frequency corresponding to 1/2 of the sampling frequency.
  • FIG. 10 shows the following figures.
  • Example A Frequency characteristics of HPF
  • the HPF characteristic shown in this graph is a characteristic in which the change becomes smaller as the frequency approaches the vicinity of the Nyquist frequency.
  • the results of the in-focus degree change analysis when the HPF having such characteristics is used as the first HPF152 and the second HPF153 in the in-focus degree change analysis unit 105 of the image pickup apparatus of the present disclosure are shown in (a2) on the right side. It is a graph.
  • This graph (a2) is a graph showing time on the horizontal axis and focusing degree (HPF result) on the vertical axis, as described above with reference to FIGS. 7 and 8.
  • the degree of focus on the vertical axis (HPF result) corresponds to the amount of high-frequency component detected in the HPF filtering result image.
  • the difference between the in-focus degree of the current image and the past image becomes smaller than the regulation threshold value, and as a result, it is determined as a pixel whose in-focus degree does not change even though the pixel is approaching in-focus. There is a possibility that it will be done.
  • the amplitude characteristics increase at an accelerating rate as the frequency approaches the high frequency side. It is preferable to use the HPF having. That is, it is preferable that the HPF has a characteristic that the filtering result of the HPF has a large slope up to the peak.
  • FIG. 11 shows the following figures.
  • B1 HPF frequency characteristic example B
  • B2 Example of frequency characteristic example Focus change analysis example when HPF having the characteristics shown in B is applied.
  • Example B Frequency characteristic of HPF
  • the HPF characteristic shown in this graph is a characteristic in which the change becomes large as the frequency approaches the vicinity of the Nyquist frequency.
  • the results of the in-focus degree change analysis when the HPF having such characteristics is used as the first HPF152 or the second HPF153 in the in-focus degree change analysis unit 105 of the image pickup apparatus of the present disclosure are shown in (b2) on the right side. It is a graph.
  • the graph of (b2) is a graph showing time on the horizontal axis and focusing degree (HPF result) on the vertical axis.
  • the degree of focus on the vertical axis (HPF result) corresponds to the amount of high-frequency component detected in the HPF filtering result image.
  • the HPF 152 used as the first HPF 152 and the HPF used as the second HPF 153 in the in-focus degree change analysis unit 105 of the image pickup apparatus of the present disclosure are HPFs having a characteristic that the amplitude characteristic increases at an accelerating rate as the frequency approaches the high frequency side. Is preferable.
  • FIG. 12 is a block diagram showing a configuration example of the in-focus degree change analysis unit 105b of the third embodiment. That is, it is a block diagram which shows the structural example of the focusing degree change analysis unit 105c which performs pixel classification using the averaging data of a plurality of pixel areas.
  • the display unit 107 has a smaller number of pixels than the number of pixels of the image sensor (image sensor) 104.
  • the image displayed on the display unit 107 needs to be an image in which the number of pixels of the image sensor output image 121, which is the output of the image sensor (image sensor) 104, is reduced, that is, a down-converted image. ..
  • the display control unit 106 in the configuration shown in FIG. 12 executes the down-conversion process of the image sensor output image 121.
  • the pixel classification information 124 output by the in-focus degree change analysis unit 105c to the display control unit 106 also corresponds to the pixel position of the down-converted image. It is necessary to use the pixel classification information.
  • the in-focus degree change analysis unit 105c shown in FIG. 12 has a configuration for realizing this process.
  • the in-focus degree change analysis unit 105c shown in FIG. 12 has a first HPF output down-conversion unit 221 and a second HPF output down-conversion unit 222 in the in-focus degree change analysis unit 105 shown in FIG. 9 described above as the second embodiment. Is added.
  • the first HPF output down-converting unit 221 is configured between the first HPF 152 and the pixel area unit first HPF output averaging unit 201.
  • the first HPF output down-conversion unit 221 executes a down-conversion process of the current image HPF filtering image output from the first HPF 152.
  • the down-conversion rate (pixel count reduction rate) of this down-conversion process is the same as the down-conversion rate of the down-conversion process executed by the display control unit 106 for the image sensor output image 121.
  • the first HPF output down-conversion unit 221 executes the down-conversion process of the current image HPF filtering image generated by the first HPF 152 to generate the down-converted image of the current image HPF filtering image.
  • the down-converted image of the current image HPF filtering image generated by the first HPF output down-converting unit 221 is input to the pixel area unit first HPF output averaging unit 201.
  • the pixel area unit first HPF output averaging unit 201 executes processing for the down-converted image of the current image HPF filtering image.
  • the second HPF output down-converting unit 222 is configured between the second HPF 153 and the pixel area unit second HPF output averaging unit 202.
  • the second HPF output down-conversion unit 222 executes the down-conversion process of the past image HPF filtering image output from the second HPF 153.
  • the down-conversion rate (pixel count reduction rate) of this down-conversion process is also the same as the down-conversion rate of the down-conversion process executed by the display control unit 106 for the image sensor output image 121.
  • the second HPF output down-conversion unit 222 executes the down-conversion process of the past image HPF filtering image generated by the second HPF 153 to generate the down-converted image of the past image HPF filtering image.
  • the down-converted image of the past image HPF filtering image generated by the second HPF output down-converting unit 222 is input to the second HPF output averaging unit 202 in pixel area units.
  • the second HPF output averaging unit 202 for each pixel area executes processing for the down-converted image of the past image HPF filtering image.
  • the pixel area unit first HPF output averaging unit 201 and the pixel area unit second HPF output averaging unit 202 are used to down-convert the current image HPF filtering image and down the past image HPF filtering image, respectively.
  • the execution result for the converted image is input.
  • All of the execution results are the results for the down-converted image of the HPF filtering image, and the pixel classification unit 154 compares the results of the down-converted image of the HPF filtering image with respect to the past image to perform pixel classification.
  • the number of pixels to be classified as pixels corresponds to the number of pixels of the down-converted image generated by the down-convert process executed by the display control unit 106 on the image sensor output image 121.
  • the display control unit 106 focuses on the down-converted image generated by the down-convert process for the image sensor output image 121 based on the pixel classification information corresponding to the number of pixels of the down-converted image output from the in-focus degree change analysis unit 105c. Pixel-based focusing degree change information indicating a degree change state is generated, and display data 125 including these data is output to the display unit 107.
  • the first HPF output down-conversion unit 221 is set between the first HPF 152 and the pixel area unit first HPF output averaging unit 201, and the second HPF output is down.
  • the conversion unit 222 is set between the second HPF 153 and the pixel area unit second HPF output averaging unit 202,
  • the down-conversion unit in the in-focus degree change analysis unit 105c can be set to various positions other than the positions shown in FIG. 12 as long as it is the position after the first HPF 152 and the second HPF 153.
  • the HPF filter image By configuring the HPF filter image to be down-converted in the in-focus degree change analysis unit in this way, even if the image displayed on the display unit is the down-converted image of the image sensor output image 121, the down-converted image is down. It is possible to output focus change information corresponding to the converted image.
  • the technology disclosed in the present specification can have the following configurations.
  • An in-focus degree change analysis unit that analyzes a change in the in-focus degree between the current image output from the image sensor and the past image output from the image sensor before the output of the current image.
  • a display control unit that uses the analysis result of the in-focus degree change analysis unit to generate display data that enables confirmation of the in-focus state from the past image in units of pixel areas of the current image.
  • An image pickup apparatus having a display unit for displaying the display data.
  • the display control unit is The image pickup apparatus according to (1), which generates display data capable of confirming the in-focus degree of the current image and the in-focus degree change state from the past image to the present image.
  • the in-focus degree change analysis unit is Each pixel, on a pixel-by-pixel basis, (A) Pixels approaching focus (b) Pixels out of focus (c) Pixels with no change in focus Pixel classification information indicating which of the above (a) to (c) is The image pickup apparatus according to (1) or (2), wherein the image is generated and output to the display control unit.
  • the display control unit is Display data that makes it possible to identify at least one of the pixels that are approaching focus, the pixels that are out of focus, and the pixels whose degree of focus does not change is generated for each pixel of the current image.
  • the image pickup apparatus according to any one of (1) to (3).
  • the display control unit is It is possible to identify at least one of pixels that are approaching focus, pixels that are out of focus, and pixels that do not change the degree of focus in a through image that is a developed image of the output of the image sensor.
  • the image pickup apparatus according to any one of (1) to (4), which generates display data in which the focus change determination signal is superimposed.
  • the display control unit is The image pickup apparatus according to any one of (1) to (6), which generates display data including a bar indicator capable of determining the degree of focus of a pixel approaching focus.
  • the in-focus degree change analysis unit is A first filter unit that generates current image in-focus degree analysis data that can identify the in-focus degree of each pixel by executing the filtering process for the current image, and A second filter unit that generates past image in-focus degree analysis data that can identify the in-focus degree of each pixel by executing the filtering process for the past image, and The current image focus analysis data is compared with the past image focus analysis data to analyze the change in focus in pixel units or pixel region units, and pixel classification information is generated based on the analysis result.
  • the image pickup apparatus according to any one of (1) to (7), which has a pixel classification unit.
  • the in-focus degree change analysis unit is It has a memory for storing the output image from the image sensor, and has a memory.
  • the first filter unit and the second filter unit are The image pickup apparatus according to (8) or (9), which executes a filtering process to which an HPF (high Pass Filter) is applied.
  • HPF high Pass Filter
  • the HPF used by the first filter unit and the second filter unit is The image pickup apparatus according to (10), which is an HPF having a characteristic that the amplitude characteristic increases at an accelerating rate as it approaches the high frequency side.
  • the in-focus degree change analysis unit is The output averaging unit of the first filter unit, which averages the output of the first filter unit in units of a plurality of pixels, It has a second filter unit output averaging unit that averages the output of the second filter unit in units of a plurality of pixels.
  • the pixel classification unit is The output of the output averaging unit of the first filter unit is compared with the output of the output averaging unit of the second filter unit to analyze the change in the degree of focus in pixel units or pixel region units, and based on the analysis result.
  • the image pickup apparatus according to any one of (8) to (11), which has a pixel classification unit for generating pixel classification information.
  • the display control unit is The configuration is such that the down-converted image generated by down-converting the output image from the image sensor is output to the display unit.
  • the focus change analysis unit is The image pickup apparatus according to any one of (1) to (12), which generates pixel classification information indicating a change in the degree of focus of each pixel of the down-converted image.
  • the in-focus degree change analysis unit is A first filter unit that generates current image in-focus degree analysis data that can identify the in-focus degree of each pixel by executing the filtering process for the current image, and A second filter unit that generates past image in-focus degree analysis data that can identify the in-focus degree of each pixel by executing the filtering process for the past image, and
  • the in-focus degree change analysis unit analyzes the change in the in-focus degree between the current image output from the image sensor and the past image output from the image sensor before the output of the current image.
  • a display control step for generating display data in which the display control unit can confirm the in-focus state from the past image in units of the pixel area of the current image by using the analysis result of the in-focus degree change analysis unit.
  • a program that executes image processing in an image processing device Focus change analysis step for causing the in-focus degree change analysis unit to analyze the change in the in-focus degree between the current image output from the image sensor and the past image output from the image sensor before the output of the current image.
  • a display control step for causing the display control unit to generate display data capable of confirming the in-focus state from the past image in units of the pixel area of the current image by using the analysis result of the in-focus degree change analysis unit.
  • the series of processes described in the specification can be executed by hardware, software, or a composite configuration of both.
  • the program can be pre-recorded on a recording medium.
  • programs can be received via networks such as LAN (Local Area Network) and the Internet, and installed on a recording medium such as a built-in hard disk.
  • the various processes described in the specification are not only executed in chronological order according to the description, but may also be executed in parallel or individually as required by the processing capacity of the device that executes the processes.
  • the system is a logical set configuration of a plurality of devices, and the devices having each configuration are not limited to those in the same housing.
  • the change in focus between the current image and the past image is analyzed, and display data that can identify the change in focus is output to the display unit.
  • the device and method to do so are realized.
  • an in-focus degree change analysis unit that analyzes a change in the in-focus degree between the current image output from the image sensor and the past image output from the image sensor before the output of the current image is used. It has a display control unit that uses the analysis result of the focus change analysis unit to generate display data that enables confirmation of the focus change state from the past image in units of pixel areas of the current image.
  • each pixel is either (a) a pixel that is approaching focus, (b) a pixel that is out of focus, or (c) a pixel that does not change the degree of focus.
  • Generate and display classification information With this configuration, a device and a method for analyzing changes in the in-focus degree between the current image and the past image and outputting display data capable of identifying the change in the in-focus degree to the display unit are realized.
  • Image pickup device 101 Input section 102 Control section 103 Drive section 104 Image sensor (image sensor) 105 Focus change analysis unit 106 Display control unit 107 Display unit 151 Memory 152 1st HPF 153 2nd HPF 154 Pixel classification unit 201 Pixel area unit 1st HPF output averaging unit 202 Pixel area unit 2nd HPF output averaging unit 221 1st HPF output down conversion unit 222 2nd HPF output down conversion unit

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Human Computer Interaction (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Computation (AREA)
  • General Health & Medical Sciences (AREA)
  • Medical Informatics (AREA)
  • Software Systems (AREA)
  • Databases & Information Systems (AREA)
  • Computing Systems (AREA)
  • Health & Medical Sciences (AREA)
  • General Engineering & Computer Science (AREA)
  • Computer Hardware Design (AREA)
  • Studio Devices (AREA)
  • Indication In Cameras, And Counting Of Exposures (AREA)
  • Automatic Focus Adjustment (AREA)
PCT/JP2021/018917 2020-06-04 2021-05-19 撮像装置、および画像処理方法、並びにプログラム Ceased WO2021246171A1 (ja)

Priority Applications (2)

Application Number Priority Date Filing Date Title
JP2022528528A JP7632463B2 (ja) 2020-06-04 2021-05-19 撮像装置、および画像処理方法、並びにプログラム
US17/921,495 US12170845B2 (en) 2020-06-04 2021-05-19 Imaging device, image processing method, and program

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2020097742 2020-06-04
JP2020-097742 2020-06-04

Publications (1)

Publication Number Publication Date
WO2021246171A1 true WO2021246171A1 (ja) 2021-12-09

Family

ID=78830458

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2021/018917 Ceased WO2021246171A1 (ja) 2020-06-04 2021-05-19 撮像装置、および画像処理方法、並びにプログラム

Country Status (3)

Country Link
US (1) US12170845B2 (https=)
JP (1) JP7632463B2 (https=)
WO (1) WO2021246171A1 (https=)

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2005114858A (ja) * 2003-10-03 2005-04-28 Jai Corporation 測距カメラ装置
JP2010219741A (ja) * 2009-03-16 2010-09-30 Nikon Corp 撮像装置および画像処理装置
JP2016024489A (ja) * 2014-07-16 2016-02-08 キヤノン株式会社 画像処理装置、撮像装置、画像処理方法及びプログラム

Family Cites Families (16)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH05127244A (ja) * 1991-11-07 1993-05-25 Nikon Corp 焦点状態を表示する表示装置
JPH06113184A (ja) * 1992-09-24 1994-04-22 Casio Comput Co Ltd カメラ装置
JP4565784B2 (ja) * 2001-09-20 2010-10-20 Ｈｏｙａ株式会社 デジタルカメラ
JP2005140943A (ja) * 2003-11-06 2005-06-02 Canon Inc フォーカスアシスト装置、ドライブユニット、撮影レンズおよび撮影システム
JP5003281B2 (ja) * 2007-05-23 2012-08-15 株式会社ニコン 焦点状態表示装置および焦点状態表示方法
JP5039570B2 (ja) * 2008-01-09 2012-10-03 キヤノン株式会社 表示装置、映像表示システム、及び表示方法
JP5088225B2 (ja) 2008-05-01 2012-12-05 ソニー株式会社 映像信号処理装置、撮像装置及び映像信号処理方法
JP6442709B2 (ja) * 2014-09-08 2018-12-26 パナソニックＩｐマネジメント株式会社 撮像装置
JP6834945B2 (ja) 2015-04-07 2021-02-24 ソニー株式会社 画像信号処理装置、画像信号処理方法および撮像装置
US10331014B2 (en) * 2015-06-18 2019-06-25 Sony Corporation Image processing apparatus, image processing method, display control apparatus, and display control method
US10341550B2 (en) * 2016-11-28 2019-07-02 Anritsu Corporation End face inspection apparatus and focused image data acquisition method
WO2021200143A1 (ja) * 2020-03-30 2021-10-07 ソニーグループ株式会社 画像処理装置、画像処理方法、および、３ｄモデルデータ生成方法
EP3996038A1 (en) * 2020-11-06 2022-05-11 Canon Kabushiki Kaisha Information processing apparatus, information processing method, and program
JP7509918B2 (ja) * 2020-12-10 2024-07-02 ファナック株式会社 画像処理システム及び画像処理方法
JP7768680B2 (ja) * 2021-02-19 2025-11-12 株式会社キーエンス 拡大観察装置、拡大画像観察方法、拡大画像観察プログラム及びコンピュータで読み取り可能な記録媒体並びに記憶した機器
WO2022244351A1 (ja) * 2021-05-17 2022-11-24 ソニーグループ株式会社 画像処理装置、画像処理方法、プログラム

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2005114858A (ja) * 2003-10-03 2005-04-28 Jai Corporation 測距カメラ装置
JP2010219741A (ja) * 2009-03-16 2010-09-30 Nikon Corp 撮像装置および画像処理装置
JP2016024489A (ja) * 2014-07-16 2016-02-08 キヤノン株式会社 画像処理装置、撮像装置、画像処理方法及びプログラム

Also Published As

Publication number Publication date
US12170845B2 (en) 2024-12-17
JP7632463B2 (ja) 2025-02-19
JPWO2021246171A1 (https=) 2021-12-09
US20230164436A1 (en) 2023-05-25

Similar Documents

Publication Publication Date Title
US8629915B2 (en) Digital photographing apparatus, method of controlling the same, and computer readable storage medium
US9489747B2 (en) Image processing apparatus for performing object recognition focusing on object motion, and image processing method therefor
US8184171B2 (en) Image pickup apparatus, image processing apparatus, image pickup method, and image processing method
CN100579179C (zh) 自动对焦装置、图像捕获装置和自动对焦方法
KR100994596B1 (ko) 소정 조건을 따른 화상을 기록하는 촬상장치, 촬영방법 및 그 프로그램을 기록한 기록매체
US10395348B2 (en) Image pickup apparatus, image processing apparatus, and control method of image pickup apparatus
KR101399012B1 (ko) 영상 복원 장치 및 방법
US8989447B2 (en) Dynamic focus for computational imaging
KR100897768B1 (ko) 자동 초점 조절 방법과 상기 방법을 사용할 수 있는 장치
US20150002684A1 (en) Image processing apparatus
JP5380784B2 (ja) オートフォーカス装置、撮像装置及びオートフォーカス方法
CN103004179B (zh) 跟踪装置和跟踪方法
KR101299249B1 (ko) 디지털 촬영장치, 그 제어방법 및 제어방법을 실행시키기 위한 프로그램을 저장한 기록매체
WO2005022901A1 (ja) 撮像系診断装置、撮像系診断プログラム、撮像系診断プログラム製品、および撮像装置
KR20100013171A (ko) 오토 포커스 영역의 움직임 보상 방법 및 장치, 이를이용한 오토 포커스 방법 및 장치
KR20170101532A (ko) 이미지 융합 방법 및 이를 위한 컴퓨터 프로그램, 그 기록매체
US7733411B2 (en) Image capturing apparatus performing filtering process with variable cut-off frequency
US7519285B2 (en) Imaging apparatus, imaging method, imaging program, and storage medium
JP2014110624A (ja) 画像処理装置、画像処理方法及びプログラム
JP7632463B2 (ja) 撮像装置、および画像処理方法、並びにプログラム
US10911660B2 (en) Control apparatus, imaging apparatus, control method, and storage medium
US10477098B2 (en) Control apparatus which sets defocus amount used for focusing, image capturing apparatus, control method, and storage medium
CN113572968A (zh) 图像融合方法、装置、摄像设备及存储介质
WO2020044844A1 (ja) 信号処理装置、信号処理方法、信号処理プログラムおよび撮像装置
EP4346223A2 (en) Display control apparatus and display control method therefor

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21817499

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2022528528

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21817499

Country of ref document: EP

Kind code of ref document: A1