JP2019121914A - Image processing device and image processing method - Google Patents

Abstract

To provide an image processing device and an image processing method capable of selecting a special image process whose effect is easy for a user to notice and which is easy to use.SOLUTION: The image processing device comprises: an image processing unit 112 for applying a special image process of imparting a special effect, to a photographed image recorded in an image file; and a display control unit 113 for displaying the photographed image subjected to the special image process on a display element 14.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an image processing apparatus and an image processing method.

  There are known processing devices that perform special image processing that applies various special effects to an image. Among such processing devices, for example, the moving image editing device of Patent Document 1 performs special image processing of overlaying a decorative image having a motion on a moving image according to the user's selection. A moving image generated as a result of such special image processing is displayed on a display device provided in the moving image editing device.

JP, 2015-35663, A

  There are many special image processings. For this reason, the user may find it difficult to notice the effects of individual special image processing. In order to notice the effects of individual special image processing, it is effective to try special image processing on various images.

  The present invention has been made in view of the above circumstances, and it is an object of the present invention to provide an image processing apparatus and an image processing method capable of selecting special image processing which is easy for the user to notice effects and easy to use.

  In order to achieve the above object, an image processing apparatus according to a first aspect of the present invention includes a recording unit, a display element, and a special effect on a first image recorded as an image file in the recording unit. And a display control unit that causes the display element to display the second image.

  In order to achieve the above object, an image processing apparatus according to a second aspect of the present invention is an image processing unit that performs special image processing for applying special effects to a captured image recorded in an image file; And a display control unit configured to display the photographed image subjected to the special image processing on a display element.

  In order to achieve the above object, in the image processing method according to the third aspect of the present invention, the image processing unit performs special image processing for applying special effects to a photographed image recorded in an image file. And displaying the photographed image subjected to the special image processing on a display element.

  According to the present invention, it is possible to provide an image processing apparatus and an image processing method in which the user can easily notice the effect and can select the special image processing with a simple operation.

FIG. 1 is a view showing the configuration of a portable device provided with an image processing apparatus according to each embodiment of the present invention. FIG. 2A is a flow chart showing the operation of the mobile device. FIG. 2B is a flowchart showing the operation of the mobile device. FIG. 3A is a diagram showing a first example of an image file. FIG. 3B is a diagram showing a second example of the image file. FIG. 4A is a diagram showing an example of reproduction of an image file. FIG. 4B is a diagram showing a first example of a selection screen for special image processing in the first embodiment. FIG. 4C is a diagram showing a second example of the selection screen for special image processing in the first embodiment. FIG. 5 is a flowchart showing the setting process of the application of the special effect in the second embodiment. FIG. 6A is a diagram showing an example of live view display in the second embodiment. FIG. 6B is a view showing an example of a selection screen for special image processing in the second embodiment. FIG. 6C is a diagram showing a display example of a sample image in the second embodiment. FIG. 6D is an example of list display. FIG. 6E is a view showing a display example of a sample image in the second embodiment.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.
First Embodiment
First, a first embodiment of the present invention will be described. FIG. 1 is a view showing the configuration of a portable device provided with an image processing apparatus according to each embodiment of the present invention. The portable device 1 illustrated in FIG. 1 includes a control circuit 11, an imaging unit 12, a memory 13, a display element 14, a recording medium 15, an operation member 16, and a communication circuit 17. The portable device 1 is, for example, a portable device having an imaging function such as a digital camera, a smartphone, a tablet terminal, or a portable game machine. The imaging unit 12 may not be necessary as long as an image captured by another device can be communicated to the mobile device 1.

  The control circuit 11 is a control circuit configured by hardware including a CPU and the like. The control circuit 11 may be configured by an ASIC, an FPGA, or the like. The control circuit 11 controls the operation of the mobile device 1. The control circuit 11 includes an imaging control unit 111, an image processing unit 112, a display control unit 113, and a recording processing unit 114. The imaging control unit 111 controls acquisition of an image by the imaging unit 12 and readout of the image. The image processing unit 112 performs various image processing on the image. The image processing unit 112 includes a normal image processing unit 112 a, a special image processing unit 112 b, and a resize processing unit 112 c. The normal image processing unit 112 a performs image processing necessary for displaying or recording an image, such as white balance correction, tone correction, color correction, etc., on the image. Further, the normal image processing unit 112a applies still image compression or moving image compression to the image. The special image processing unit 112 b applies image processing such as blurring processing and shading processing to the image to give a special visual effect to the image. The resize processing unit 112 c performs image resizing (reduction processing). The display control unit 113 controls the display of an image by the display element 14. The recording processing unit 114 controls recording of an image on the recording medium 15. The imaging control unit 111, the image processing unit 112, the display control unit 113, and the recording processing unit 114 may be configured as dedicated circuits. Further, the imaging control unit 111, the image processing unit 112, the display control unit 113, and the recording processing unit 114 may be configured by an integrated circuit or the like in which part or all of them are combined. Furthermore, functions equivalent to those of the imaging control unit 111, the image processing unit 112, the display control unit 113, and the recording processing unit 114 may be realized using software processing.

  The imaging unit 12 includes a lens, an imaging element, and a preprocessing circuit. The lens focuses a light flux from an object (not shown) on the light receiving surface of the imaging device. The imaging element outputs an image signal by imaging a subject. The imaging device is an imaging device such as a CCD sensor or a CMOS sensor. The preprocessing circuit performs preprocessing on the image signal. Preprocessing includes, for example, correlated double sampling, gain adjustment, and analog-to-digital conversion.

  The memory 13 temporarily stores various data. The memory 13 is, for example, a memory such as a DRAM and an SDRAM. The memory 13 may be configured by a single memory or may be configured by a plurality of memories.

  The display element 14 displays various images. The display element 14 is, for example, a display element such as a liquid crystal display and an organic EL display. On the display screen of the display element 14, a touch panel as one of the operation members 16 may be formed.

  Various programs executed by the control circuit 11 are recorded in the recording medium 15. Further, an image file is recorded on the recording medium 15. The image file is a file recorded with the image compressed (or resized). The recording medium 15 is a memory such as a flash memory, for example. The recording medium 15 may be configured by a single memory or may be configured by a plurality of memories.

  The operation member 16 includes various operation members for the user to perform various operations on the portable device 1. The operating member 16 includes mechanical operating members such as buttons and switches. In addition, the operation member 16 includes a touch panel.

  The communication circuit 17 includes a circuit for the portable device 1 to communicate with other devices. The communication may be performed by either wireless communication such as Wi-Fi (registered trademark) or wired communication such as USB. Furthermore, if the recording medium 15 is a removable recording medium, communication may be performed via the recording medium 15.

  Hereinafter, the operation of the mobile device 1 will be described. 2A and 2B are flowcharts showing the operation of the mobile device 1. The operations of FIGS. 2A and 2B are mainly controlled by the control circuit 11. The operation of FIG. 1 is started, for example, when the power of the portable device 1 is turned on. In step S1, the control circuit 11 determines whether the operation mode of the mobile device 1 is the shooting mode. The portable device 1 has at least a shooting mode, a reproduction mode, and a communication mode as operation modes. The shooting mode is an operation mode for shooting an image. The reproduction mode is an operation mode for reproducing an image file recorded on the recording medium 15 or the like. The communication mode is an operation mode for communication of an image file or the like. The portable device 1 may have an operation mode other than these. Selection of the operation mode is performed, for example, by operating the operation member 16. When it is determined in step S1 that the operation mode is the shooting mode, the process proceeds to step S2. If it is determined in step S1 that the operation mode is not the shooting mode, the process proceeds to step S18.

  In step S2, the control circuit 11 causes the imaging unit 12 to perform imaging for live view display. In response to this, the imaging unit 12 images an object. The image obtained by the imaging unit 12 is digitized and stored in the memory 13. When there is an imaging device outside the mobile device 1, the control circuit 11 instructs the imaging device outside the mobile device 1 to perform imaging for live view display. Then, the control circuit 11 receives an image from this external imaging device.

  At step S3, the control circuit 11 determines whether or not the image is set to be applied with a special effect. When it is instructed to set a special effect on the image in step S7 described later, it is set to apply the special effect to the image. If it is determined in step S3 that the special effect is to be applied to the image, the process proceeds to step S4. When it is not set in step S3 that the special effect is to be applied to the image, the process proceeds to step S5. In the determination of step S3 at the time of displaying the live view of the first frame, it is determined that the image is not set to be applied with the special effect.

  In step S4, the control circuit 11 performs normal image processing necessary for live view display, such as white balance correction and color correction, on the image obtained by the imaging of the imaging unit 12. Furthermore, the control circuit 11 performs special image processing set by the user on the image to which normal image processing has been applied. Thereafter, the process proceeds to step S6.

  In step S5, the control circuit 11 performs normal image processing necessary for live view display, such as white balance correction and color correction, on the image obtained by the imaging of the imaging unit 12. Thereafter, the process proceeds to step S6.

  In step S6, the control circuit 11 performs live view display on the display element 14. In step S6, the control circuit 11 inputs the image processed in step S4 or step S5 to the display element 14. The display element 14 displays the input image. As a result, on the display element 14, an image to which a special effect is applied or an image to which a special effect is not applied is displayed in live view in real time. Thereafter, the process proceeds to step S7.

  In step S7, the control circuit 11 determines whether the user has instructed to apply a special effect. For example, it is determined that the user is instructed to apply the special effect when the operation member for instructing to apply the special effect is operated by the user or when the user is instructed to apply the special effect by the operation of the touch panel. Ru. When it is determined in step S7 that the application of the special effect is instructed by the user, the process proceeds to step S8. When it is determined in step S7 that the application of the special effect is not instructed by the user, the process proceeds to step S9.

  In step S8, the control circuit 11 sets the application of the special effect. Thereafter, the process proceeds to step S9. Thus, special image processing is performed on the image during the next live view display.

  In step S9, the control circuit 11 determines whether to shoot. For example, when the operation member for instructing imaging is operated by the user or when imaging is instructed by the operation of the touch panel of the user, it is determined that imaging is performed. If it is determined in step S9 that shooting is to be performed, the process proceeds to step S10. If it is determined in step S9 that no shooting is to be performed, the process proceeds to step S17.

  In step S10, the control circuit 11 causes the imaging unit 12 to perform imaging for recording. In response to this, the imaging unit 12 images an object. The image obtained by the imaging unit 12 is digitized and stored in the memory 13. Alternatively, an image received from an external imaging device is stored in the memory 13. Note that the shutter speed and the aperture value in imaging for recording may be determined by a known photometric process.

  In step S11, the control circuit 11 determines whether to end the imaging. For example, in the case of still image shooting, it is determined that the shooting is ended after the end of one shooting. In addition, for example, in the case of continuous shooting, it is determined that shooting is ended after completion of the specified number of times of shooting. Furthermore, for example, in the case of a moving image, when the user instructs to end moving image shooting, it is determined that the shooting is ended. If it is determined in step S11 that the shooting is to be ended, the process proceeds to step S12. If it is determined in step S11 that the photographing is not finished, the process returns to step S10. In this case, shooting is continued.

  In step S12, the control circuit 11 generates a RAW resized image from the image stored in the memory 13. The RAW resized image is generated by resizing the image stored in the memory 13 to a predetermined size. That is, the RAW resized image is a resized image of the image obtained by the imaging unit 12 and is an image to which neither the normal image processing nor the special image processing is applied.

  In step S13, the control circuit 11 performs image processing on the image stored in the memory 13. That is, the control circuit 11 performs normal image processing necessary for live view display, such as white balance correction and color correction, on an image obtained by imaging of the imaging unit 12. Furthermore, when the setting of the application of the special effect is performed, the control circuit 11 performs the special image processing set by the user on the image on which the normal image processing has been performed.

  In step S14, the control circuit 11 generates a compressed image based on the image subjected to the image processing. Still image compression is used for still images, and moving image compression is used for moving images.

  In step S15, the control circuit 11 generates a thumbnail image. That is, the control circuit 11 generates a thumbnail image by reducing the image subjected to the image processing to an appropriate size.

  In step S16, the control circuit 11 generates an image file. Then, the control circuit 11 records the image file on the recording medium 15. Thereafter, the process proceeds to step S17.

  FIG. 3A is a diagram showing a first example of an image file generated in the present embodiment. As shown in FIG. 3A, the image file has a RAW resized image D1, a compressed image D2, a thumbnail D3, and image information D4. The RAW resized image D1 is data of the RAW resized image generated in step S12. The compressed image D2 is data of the compressed image generated in step S13. The thumbnail D3 is a thumbnail generated in step S14. The image information D4 is information representing conditions for shooting. The image information D4 includes, for example, information of shooting date, shutter speed, and aperture. Further, when the addition of the special effect is set, the image information D4 includes the information of the content of the special image processing. The image information D4 may include information other than these.

  By recording the RAW resized image D1 in the image file, it is possible to re-apply an arbitrary special effect to the image to be reproduced when the image file is reproduced. Details will be described later.

  Now return to the description of FIGS. 2A and 2B. In step S17, the control circuit 11 determines whether the power of the portable device 1 is to be turned off. For example, when the power switch of the portable device 1 is turned off by the user, it is determined that the power is turned off. Besides, it may be determined that the power is turned off when there is no operation for a predetermined time. When it is determined in step S17 that the power supply of the mobile device 1 is turned off, the processing of FIGS. 2A and 2B ends. When it is determined in step S17 that the power of the portable device 1 is not turned off, the process returns to step S1.

  When it is determined in step S1 that the operation mode is not the shooting mode, in step S18, the control circuit 11 determines whether the operation mode of the portable device 1 is the reproduction mode. When it is determined in step S18 that the operation mode is the reproduction mode, the process proceeds to step S19. When it is determined in step S18 that the operation mode is not the reproduction mode, that is, the communication mode, the process proceeds to step S29.

  In step S19, the control circuit 11 causes the display element 14 to display a list of image files (captured images) recorded in the recording medium 15. The list display is performed, for example, by displaying thumbnails.

  In step S20, the control circuit 11 determines whether an image file has been selected by the user. For example, selection of an image file is performed using a touch panel. If it is determined in step S20 that an image file has been selected, the process proceeds to step S21. When it is determined in step S20 that the image file is not selected, the process proceeds to step S17. In this case, when it is not determined in step S17 that the power is turned off, the list display is continued.

  In step S <b> 21, the control circuit 11 reads the selected image file from the recording medium 15 and stores it in the memory 13. Thereafter, the control circuit 11 determines whether or not the image is set to be applied with a special effect. When it is instructed to set a special effect on the image in step S24 described later, it is set to apply the special effect to the image. If it is determined in step S21 that the special effect is to be applied to the image, the process proceeds to step S22. In step S21, when it is not set to apply the special effect to the image, the process proceeds to step S23. In the determination at step S21 after the image file to be reproduced is selected or after the image file to be reproduced is selected, it is determined that the image is not set to be applied with a special effect.

  In step S22, the control circuit 11 performs normal image processing on the RAW resized image recorded in the read image file. At this time, the control circuit 11 may also perform processing for enlarging the RAW resized image according to the display characteristics of the display element 14. Furthermore, the control circuit 11 performs special image processing set by the user on the image to which normal image processing has been applied. Thereafter, the process proceeds to step S23. That is, in the present embodiment, when it is set that the special effect is given at the time of reproduction of the image file, the special image processing is performed on the RAW image, not on the decompression result of the compressed image. In this way, the applied special image processing is not affected by another special image processing.

  In step S23, the control circuit 11 reproduces an image on the display element 14. When the special image processing is performed in step S22, the control circuit 11 inputs the image processed in step S22 to the display element 14. In addition, when the special image processing is not performed in step S22, the control circuit 11 decompresses the compressed image recorded in the read image file. Then, the control circuit 11 inputs the expanded image to the display element 14. The display element 14 displays the input image. As a result, an image to which the special effect is applied or an image to which the special effect is not applied is reproduced on the display element 14. Thereafter, the process proceeds to step S24.

  FIG. 4A is a diagram showing an example of reproduction of an image file. As shown in FIG. 4A, at the time of reproduction of the image file, the reproduced image I is displayed on the display element 14. Furthermore, in the present embodiment, the processing button B1 is displayed on the display element 14. The processing button B1 is a software button operated by the touch panel. The processing button B1 is a button selected for instructing special image processing. At the time of reproduction of the image file, other information such as the imaging condition of the image may be displayed together.

  Now return to the description of FIGS. 2A and 2B. In step S24, the control circuit 11 determines whether the user has instructed to apply a special effect. Hereinafter, an example of selection of special image processing in the reproduction mode will be described.

  For example, when the processing button B1 is selected by the user on the screen shown in FIG. 4A, the control circuit 11 displays a selection screen for special image processing. FIG. 4B is a diagram showing a first example of a selection screen for special image processing. On the special image processing selection screen of the first example, a selection button B2 and a sample image SI are displayed. The selection button B2 has a feed button and a return button. The send button is a button for instructing to change the selected special image process to the process after one. The return button is a button for instructing to change the selected special image processing to the previous processing. The sample image SI is an image showing an application example of the special image processing (processing A in the figure) being selected. The sample image SI displayed first is an image recorded in advance on the recording medium 15.

  The user finds a desired special image processing by looking at the sample image SI while selecting the selection button B2 on the selection screen of the special image processing. Then, when the user finds a desired special image process, the user selects, for example, a sample image SI. At this time, in step S24, it is determined that the application of the special effect is instructed by the user. When it is determined in step S24 that the application of the special effect is instructed by the user, the process proceeds to step S25. When it is determined in step S24 that the application of the special effect is not instructed by the user, the process proceeds to step S26.

  FIG. 4A is an example in which one sample image SI of special image processing is displayed. On the other hand, sample images of a plurality of special image processes may be displayed at one time. FIG. 4C is a diagram showing a second example of a selection screen for special image processing. In the selection screen for special image processing of the second example, sample images SI1 and SI2 for each of a plurality of special image processing (in the figure, processing A and processing B) are displayed. The user finds the desired special image processing by comparing the sample images SI1 and SI2 on the special image processing selection screen. Then, the user selects a desired special image processing sample image. At this time, in step S24, it is determined that the application of the special effect is instructed by the user. In FIG. 4C, sample images of two types of special image processing are displayed at one time. On the other hand, sample images of three or more types of special image processing may be displayed at one time. Furthermore, there may be a case where there are many special image processing sample images and it is not possible to display all the sample images at one time. The sample image which can not be displayed may be displayed by, for example, a scroll operation of the screen.

  In step S25, the control circuit 11 sets the application of the special effect. Thereafter, the process proceeds to step S26. Thereby, in the determination of the next step S21, it is determined that the application of the special effect is set. Then, in step S22, special image processing is performed. Then, in step S23, the image to which the special effect has been applied is reproduced.

  In step S26, the control circuit 11 determines whether to change the image to be reproduced. For example, it is determined that the image to be reproduced is to be changed when the operation member for instructing the change of the image is operated by the user or when the change of the image is instructed by the operation of the touch panel of the user. If it is determined in step S26 that the image to be reproduced is to be changed, the process proceeds to step S27. If it is determined in step S26 that the image to be reproduced is not changed, the process proceeds to step S27.

  In step S27, the control circuit 11 changes the image to be reproduced according to the user's operation. That is, the control circuit 11 reads from the recording medium 15 an image file including the image after the change. Thereafter, the process proceeds to step S28.

  In step S28, the control circuit 11 determines whether to end the reproduction of the image. For example, it is determined that the reproduction of the image is ended when the operation member for instructing the end of the reproduction of the image is operated by the user or when the end of the reproduction of the image is instructed by the operation of the touch panel of the user. If it is determined in step S28 that the reproduction of the image is not ended, the process returns to step S21. When it is determined in step S28 that the reproduction of the image is to be ended, the process proceeds to step S17.

  In step S29 when it is determined in step S18 that the operation mode is not the reproduction mode, the control circuit 11 performs the operation in the communication mode. In the communication mode, data such as an image file is transmitted to an external server or the like according to the user's operation. In the communication mode, data such as an image file is received from an external server or the like in accordance with the user's operation. After the end of the communication mode, the process proceeds to step S17.

  As described above, according to the present embodiment, it is possible to confirm an image obtained by performing special image processing on a captured image. This allows the user to notice how to use special image processing that he did not notice at the time of shooting. Also, the user can reflect how to use the special image processing that has been newly known in the next shooting. Furthermore, the user may simplify operations such as applying a plurality of special image processes to one captured image or applying a single special image process to a plurality of captured images. it can. In this way, the user can easily notice the effect and can select the special image processing with a simple operation.

  Further, in the present embodiment, a RAW resized image is recorded in the image file together with the compressed image. A RAW image is an image that has not been subjected to image processing. The thumbnail is a reduced image of the main image, and is used as a sample or index of the original image. Thumbnails are recorded corresponding to image files. Therefore, the thumbnail display allows the user to grasp the contents without opening the original image, and can easily move and organize the images. In the case of ordinary thumbnails, YC conversion data (which has already been subjected to image processing and compression processing) separately from RAW data from the sensor (YC method is calculated using conversion formulas based on values expressed in three primary colors RGB of light) The color is represented by the following numerical value, Y is the luminance, C is the hue and saturation of the color, and is embedded as part of the image file for photographing confirmation. Since specific image processing has already been performed on the YC converted thumbnails, it is necessary for the user to always re-execute image processing from the RAW image in order to confirm the edited image of the RAW image. It takes a lot of time to process an image from a RAW image with a large file size, which impairs usability when confirming a smooth editing result. In many cases, the problem of image processing speed is often solved by using a dedicated image processing engine, high-performance graphic board, etc., but the cost and size of the image processing system have major disadvantages. Therefore, a RAW image reduced simultaneously with the RAW image (or sequentially) as an image output from the sensor is recorded as an image file separately from the thumbnail, and the reduced RAW image is processed in the image processing at the time of image editing. By using it, it is possible to achieve both speeding up of the speed and reproducibility of the image processing. Therefore, the user can confirm the effect of the image processing accurately with high tempo. In addition, as a creation method of the reduced RAW image, it is general that a reduction means by pixel addition processing or thinning processing is used. Therefore, even if the special image processing is performed at the time of shooting, the special image processing at the time of shooting does not have an influence at the time of applying the special image processing at the time of reproduction. Thereby, the user can know the single effect of the selected special image processing. Also, in general, a RAW image has a large data volume. In the present embodiment, since the RAW image is resized and recorded, it is possible to suppress an increase in the data capacity more than necessary.

  In addition to the main image, in addition to the thumbnail (small image 360 × 240 size for playback), the image file is a small image only for playback (1024 × 768 size) called a screen nail that is easier to enlarge than the thumbnail. May be included. This screen nail may be used in place of the "thumbnail" of this embodiment. That is, the thumbnail may be used in place of the pixel addition or thinning out of this "screening nail" or using a small RAW image as the fourth image. Furthermore, even if it is not a RAW image obtained by strictly thinning out raw data of each pixel or adding adjacent pixels, the size of the image is small if it is a reduced image such as a thumbnail image with a reduced amount of pixel information. You don't have to make it a RAW image. That is, even if it is compressed, it is expanded to the display size, so it is possible to treat this expanded data as RAW data and handle it. Of course, the RAW data created by reducing the processing from the captured image is often held without processing the original information, just as thinning out the image data of the captured image. Even if such an image is resized and made smaller, the significance in the file is significant. That is, the effects of various image processing can be applied here by almost the same processing as image processing applied to a photographed image, the accuracy of simulation for confirming the effect in photographing is improved, and the accuracy and processing of photographing similar photographs at the time of photographing It is because the sex is good.

  Here, in the above-described step S22, special image processing is always performed on the RAW image. On the other hand, the user may be able to select whether to perform special image processing on a RAW image or special image processing on a decompression result of a compressed image. There are also cases where special image processing has already been performed on a captured image. In this case, another special image processing may be applied to the image subjected to the special image processing as it is, or another special image processing may be performed after the image in a state where the special image processing is not applied is restored. May be applied.

  Further, in the processing of FIGS. 2A and 2B, the RAW resized image and the compressed image are generated regardless of whether the special image processing is performed. On the other hand, the RAW resized image may not be generated when the special image processing is not performed.

  Further, instead of the RAW resized image, an image subjected to only normal image processing may be generated. FIG. 3B is a diagram showing a second example of the image file. The image file in the example of FIG. 3B includes a normal image-processed resized image D5, a compressed image D2, a thumbnail D3, and image information D4. The compressed image D2, the thumbnail D3, and the image information D4 are the same as in FIG. 3A. The normal image-processed resized image D5 is data of a resized image of an image subjected to only normal image processing. Even when such a normal-image-processed resized image D5 is recorded, it is possible to re-apply an arbitrary special effect to the image to be reproduced when the image file is reproduced.

  Further, in the reproduction mode of the above-described embodiment, the image to which the special effect image is applied is only displayed on the display element 14 and is not recorded on the recording medium 15. On the other hand, also in the reproduction mode, an image file including an image to which a special effect image has been applied may be recorded on the recording medium 15. In this case, the original image file may be left or deleted.

Second Embodiment
Next, a second embodiment of the present invention will be described. In the first embodiment, the special image processing can be performed on the photographed image only in the reproduction mode. On the other hand, the second embodiment is an example in which special image processing can be performed on a captured image even during live view display. The configuration of the portable device is the same as that shown in FIG. Also, the operation of the portable device is basically the same as that shown in FIGS. 2A and 2B. Specifically, the operations of steps S1-S7 and S9-S29 are the same. In the following, the operation of step S8 which is a part different from the first embodiment will be mainly described.

  FIG. 6A is a diagram showing an example of live view display in the second embodiment. As shown in FIG. 6A, the live view image LV is displayed on the display element 14 at the time of live view display. Furthermore, a processing button B1 is also displayed on the display element 14 for instructing special image processing. For example, when the process button B1 is selected by the user on the screen shown in FIG. 6A, the control circuit 11 displays a selection screen for special image processing similar to that in the reproduction mode of the first embodiment. FIG. 6B is a diagram showing an example of a selection screen for special image processing. FIG. 6B is a display of the first example. However, the display of the second example may be performed. When special image processing is selected by the user on the selection screen for special image processing, the process proceeds to step S8. Then, the setting process for applying the special effect in the second embodiment is performed. FIG. 5 is a flowchart showing the setting process of the application of the special effect in the second embodiment.

  In step S101, the control circuit 11 causes the display element 14 to display a sample image based on the live view. Specifically, the control circuit 11 reads an image stored in the memory 13 as a result of imaging for live view display. Then, the control circuit 11 performs normal image processing on the read image. At this time, the control circuit 11 also performs a process of resizing the image according to the size to be displayed as a sample image. Furthermore, the control circuit 11 performs special image processing set by the user on the image to which normal image processing has been applied. Then, the control circuit 11 inputs the image subjected to the special image processing to the display element 14. The display element 14 displays the input image. As a result, on the display element 14, a live view image to which a special effect is applied is displayed. Thereafter, the process proceeds to step S102.

  FIG. 6C is a diagram showing a display example of a sample image in the second embodiment. As shown in FIG. 6C, in the second embodiment, when the sample image SI is displayed, an import button B3 and a return button B4 are also displayed. The import button B3 is a button for instructing to perform special image processing on a photographed image. The return button B4 is a button for instructing to return to the live view display.

  Here, it returns to the explanation of FIG. In step S102, the control circuit 11 determines whether the return button B4 is selected. When it is determined in step S102 that the return button B4 is selected, the process of FIG. 5 ends. In this case, the process proceeds to step S9 of FIG. 2A. When it is determined in step S102 that the return button B4 is not selected, the process proceeds to step S103.

  In step S103, the control circuit 11 determines whether the import button B3 is selected. When it is determined in step S103 that the import button B3 is selected, the process proceeds to step S104. If it is determined in step S103 that the import button B3 is not selected, the process returns to step S102. In this case, the display of FIG. 6C is continued.

  In step S104, the control circuit 11 causes the display element 14 to display a list including the live view image and the reduced image of the image file (captured image) recorded in the recording medium 15. FIG. 6D is an example of list display. In FIG. 6D, the live view image LV is a reduced image of the live view image before the special image processing. The photographed image I is a thumbnail of each image file. In the list display, the back button B5 is also displayed. The return button B5 is a button for instructing to return the screen of FIG. 6D to the screen of FIG. 6C. Also, it is conceivable that there are a large number of photographed images I and all of the photographed images I can not be displayed at one time. The captured image I which can not be displayed may be displayed by, for example, a scroll operation of the screen. In addition, when there is an image to which special image processing has already been applied, a reduced image of the image to which special image processing has been applied may be displayed.

  In step S105, the control circuit 11 determines whether the live view image LV is selected. When it is determined in step S105 that the live view image LV is not selected, the process proceeds to step S106. When it is determined in step S105 that the live view image LV is selected, the process proceeds to step S109.

  In step S106, the control circuit 11 determines whether the photographed image I has been selected. When it is determined in step S106 that the photographed image I is selected, the process proceeds to step S107. When it is determined in step S106 that the photographed image I is not selected, the process proceeds to step S108.

  In step S107, the control circuit 11 reads out from the recording medium 15 an image file corresponding to the photographed image I selected by the user. Then, the control circuit 11 performs normal image processing on the RAW resized image recorded on the recording medium 15. At this time, the control circuit 11 also performs a process of enlarging the RAW resized image according to the size to be displayed as a sample image. Furthermore, the control circuit 11 performs special image processing set by the user on the image to which normal image processing has been applied. Thereafter, the process returns to step S101. In this case, as shown in FIG. 6E, the result of performing the special image processing on the selected captured image I is displayed on the display element 14 as a sample image SI.

  In step S108, the control circuit 11 determines whether the back button B5 is selected. If it is determined in step S108 that the return button B5 is not selected, the process returns to step S104. In this case, the display of FIG. 6D is continued. When it is determined in step S108 that the return button B5 is not selected, the process proceeds to step S101. In this case, the display of FIG. 6D returns to the display of FIG. 6C.

  In step S109 when it is determined in step S105 that the live view image LV is selected, the control circuit 11 sets the addition of the special effect. Thereafter, the process of FIG. 5 ends. In this case, the process proceeds to step S9 of FIG. 2A. For this reason, in the determination of the next step S3, it is determined that the application of the special effect is set. Then, special image processing is performed in step S4. Then, in step S6, the image to which the special effect is applied is displayed in live view.

  As described above, according to the present embodiment, it is possible to check an image obtained by performing special image processing on a captured image even during live view display. Thereby, the user can notice how to use the special image processing even during shooting. Then, the user can reflect how to use the special image processing that has been newly known to the current shooting.

  Here, the various modifications described in the first embodiment are also applied to the second embodiment. For example, in the second embodiment as well, the user may be able to select whether to perform special image processing on a RAW image or to perform special image processing on a decompression result of a compressed image.

  Although the present invention has been described above based on the embodiments, the present invention is not limited to the above-described embodiments, and it goes without saying that various modifications and applications can be made within the scope of the present invention. For example, the technology of the present embodiment can be applied to an endoscope apparatus, a microscope apparatus, a surveillance camera, an on-vehicle camera, and the like. That is, the technology of the present embodiment can be applied to various devices provided with an image processing apparatus capable of performing special image processing.

  Also, in the explanation of each operation flowchart described above, the operation is described using “first”, “next”, etc. for convenience, but it means that it is essential to carry out the operation in this order. Absent.

  In the embodiments, the parts described as “parts” (sections or units) may be configured by combining dedicated circuits or a plurality of general-purpose circuits, and if necessary, pre-programmed software It may be configured by combining a microcomputer that operates in accordance with the above, a processor such as a CPU, or a sequencer such as an FPGA. In addition, it is possible to design such that an external device takes over all or part of the control, and in this case, a wired or wireless communication circuit intervenes. The communication may be performed by Bluetooth (registered trademark), Wi-Fi, a telephone line or the like, and may be performed by USB or the like. A dedicated circuit, a general-purpose circuit, and a control unit may be integrally configured as an ASIC.

  A mechanical drive unit or the like used for auto-focusing and the like is composed of various actuators and a connection mechanism for movement as required, and the actuator is operated by a driver circuit. The drive circuit is also controlled by a microcomputer, an ASIC or the like in accordance with a specific program. Such control may be performed with detailed correction, adjustment, and the like according to information output from various sensors and their peripheral circuits.

  Also, each process according to the above-described embodiment can be stored as a program that can be executed by the control circuit 11 that is a computer. Besides, it can be stored and distributed in a storage medium of an external storage device such as a magnetic disk, an optical disk, a semiconductor memory and the like. Then, the control circuit 11 can read the program stored in the storage medium of the external storage device, and can execute the above-described processing by controlling the operation by the read program.

  The present invention is not limited to the above embodiment, and can be variously modified in the implementation stage without departing from the scope of the invention. In addition, the embodiments may be implemented in combination as appropriate, in which case the combined effect is obtained. Furthermore, various inventions are included in the above-described embodiment, and various inventions can be extracted by a combination selected from a plurality of disclosed configuration requirements. For example, even if some configuration requirements are deleted from all the configuration requirements shown in the embodiment, the problem can be solved, and if an effect is obtained, a configuration from which this configuration requirement is removed can be extracted as the invention.

  DESCRIPTION OF SYMBOLS 1 portable device 11 control circuit 12 imaging unit 13 memory 14 display element 15 recording medium 16 operation member 17 communication circuit 111 imaging control unit 112 image processing unit 112 a normal image processing unit 112 b special image Processing unit, 112 c resize processing unit, 113 display control unit, 114 recording processing unit.

Claims (9)

A recording unit,
A display element,
An image processing unit that performs special image processing that applies a special effect to a first image recorded as an image file in the recording unit to generate a second image;
A display control unit that causes the display element to display the second image;
An image processing apparatus equipped with   The image processing apparatus according to claim 1, wherein a third image is recorded in a compressed form in the image file, and the first image is recorded as a thumbnail of the third image. An image processing unit that performs special image processing for applying special effects to a captured image recorded in an image file;
A display control unit which causes the display element to display the photographed image subjected to the special image processing;
An image processing apparatus equipped with The image file includes at least the photographed image not subjected to special image processing,
The image processing apparatus according to claim 3, wherein the image processing unit performs the special image processing on the captured image not subjected to the special image processing.   The image processing apparatus according to claim 4, wherein the photographed image not subjected to the special image processing is a RAW resized image in which only resizing is performed on an image obtained by an imaging unit.   The image processing apparatus according to claim 4, wherein the photographed image which has not been subjected to the special image processing is a resized image to which only normal image processing necessary for display or recording is applied. The image file includes a compressed image of the captured image,
The image processing apparatus according to claim 3, wherein the image processing unit performs the special image processing on a result of decompressing the compressed image. The display control unit performs live view display in which the live view image obtained by the imaging unit is displayed on the display element in real time,
The image processing unit performs the special image processing on one of the live view image and the captured image according to a user's instruction during the live view display.
The display control unit includes the live view image subjected to the special image processing, the photographed image not subjected to the special image processing, the live view image subjected to the special image processing, and the special image processing. The captured image, the live view image without the special image processing and the captured image without the special image processing, or the live view image without the special image processing and the live image The image processing apparatus according to claim 3, wherein the photographed image subjected to the special image processing is displayed as a list on the display element. Performing special image processing for applying special effects to the photographed image recorded in the image file by the image processing unit;
Displaying the photographed image subjected to the special image processing on a display element;
An image processing method comprising the