JP2001230947A - Device and method for processing image - Google Patents

Abstract

PROBLEM TO BE SOLVED: To solve a problem where it is extremely complicated to set a concerned range (ROI) each time of photographing even though it is easy to set the ROI, while utilizing a viewer, for example, since how to set the ROI becomes a problem for a digital camera or the like, even though a JPEG2000 has a function for compressing an image within a certain ROI inside the image with compressibility different from that of the other area. SOLUTION: The image of an object is displayed, while overlapping a photographing guidance frame on an image display part 28 or the like for displaying the image of the object and an area within the photographing guidance frame of the photographed image is compressed by a compressing/ extending part 32 on conditions different from the area outside that frame.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image processing apparatus and method, and more particularly, to an image processing apparatus and method for photographing and recording or reproducing an image.

[0002]

2. Description of the Related Art JPEG baseline image compression is frequently used in digital cameras and the like. The JPEG baseline method uses a uniform quantization table for the entire image. For this reason, a certain area in an image cannot be compressed with higher image quality than another, and if a certain area is to be compressed with high image quality, other parts are also compressed with high image quality. As a result, the size of the compressed image file increases, and it takes time to transfer and record the image file.

A so-called JPEG2000 is being studied as a next-generation compression method. A detailed description of JPEG2000 is omitted, but a region of interest (ROI) in the image
The ROI image has a characteristic function of compressing the ROI image with a different compression coefficient (compression ratio) from other regions. This function is one way to solve the above problem.

[0004]

However, how to set the ROI with a digital camera or the like is a problem. For example, it is easy to set the ROI using the viewer, but setting the ROI for each shot will be very troublesome for the photographer.

[0005] On the other hand, there is a digital camera which can perform photographing while superimposing a photographing guidance frame and a template image on a subject image on a viewer. For example, in the case of portrait shooting, by superimposing and displaying a contour suitable for a bust shot of a person as a shooting guidance frame on a subject, the photographer can easily perform framing that is preferable as a portrait. Further, by photographing while superimposing and displaying the template image on the subject, the photographer can photograph while observing the superimposition result of the template image. Note that a mode in which shooting is performed while a guidance frame is displayed is referred to as a “guidance shooting mode”, and a mode in which shooting is performed while a template image is displayed in a superimposed manner is referred to as a “template image combining shooting mode”.

If the ROI can be set using such guidance frame and template image setting information, it is possible to save the photographer the trouble of setting the ROI.

An object of the present invention is to solve the above-mentioned problem, and an object of the present invention is to provide an image processing apparatus and a method thereof that can easily set a region of interest in a captured image.

It is another object of the present invention to provide an image processing apparatus and an image processing method for performing image processing according to a region of interest set in an image.

[0009]

The present invention has the following configuration as one means for achieving the above object.

An image processing apparatus according to the present invention is an image processing apparatus having an image capturing means for capturing an image, a display means for displaying an image of a subject, a processing means for processing the captured image, and a control means for controlling them. The control means causes the display means to display a frame for guiding the shooting on the image of the subject so as to be superimposed on the image of the subject, and causes the processing means to set the area inside the frame of the shot image as the area outside the frame. It is characterized by processing under different conditions.

An image processing apparatus comprising an image pickup means for photographing an image, a display means for displaying an image of a subject, a processing means for processing the photographed image, and a control means for controlling them. And displaying the template image on the image of the subject on the display unit, and causing the processing unit to process a region other than the template image region of the photographed image under a condition different from the template image region. And

An image processing apparatus having reading means for reading a data file stored in a memory, reproducing means for reproducing an image from the read data fail, display means for displaying an image, and control means for controlling them. The control means, when the information indicating the attention area of the image is included in the data file, causes the display means to display an image indicating the attention area on a reproduction image.
Alternatively, predetermined image processing is selectively performed on the region of interest or outside the region.

According to the image processing method of the present invention, the display means for displaying the image of the subject displays a frame for guiding the photographing on the image of the subject so as to be superimposed on the image of the subject. Is processed under a condition different from that of the area outside the frame.

The display means for displaying the image of the subject displays a template image on the image of the subject so that the area outside the template image area of the image photographed by the imaging means is different from the template image area. It is characterized by processing under conditions.

The data file stored in the memory is read, and an image is reproduced from the read data file. When the data file includes information indicating a region of interest of the image, the region of interest is superimposed on the reproduced image. Is displayed, or predetermined image processing is selectively performed on the region of interest or outside the region of interest.

[0016]

DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, an image processing apparatus according to an embodiment of the present invention will be described in detail with reference to the drawings.

[0017]

First Embodiment Hereinafter, as a first embodiment, an example in which the ROI is automatically set in the guidance shooting mode will be described.

[Configuration of Electronic Camera] FIG. 1 is a block diagram showing a configuration example of an electronic camera 100 according to the first embodiment.

In FIG. 1, reference numeral 10 denotes a photographing lens, 12 denotes a shutter having an aperture function, 14 denotes an image sensor that converts an optical image into an electric signal, and 16 denotes an analog signal that converts an analog signal output of the image sensor 14 into a digital signal. It is a digital (A / D) converter.

Reference numeral 18 denotes a timing generation circuit for supplying a clock signal and a control signal to the image pickup device 14, the A / D converter 16 and the D / A converter 26, and includes a memory control circuit 22 and a system control circuit.
Controlled to 50.

Reference numeral 20 denotes an image processing circuit which performs predetermined pixel interpolation processing and color conversion processing on data output from the A / D converter 16 or data transmitted from the memory control circuit 22. Further, the image processing circuit 20 performs a predetermined arithmetic process on data of a captured image. Based on the obtained calculation result, the system control circuit 50 controls the exposure control unit 40 and the distance measurement control unit 42 to perform TTL (Through The Lens) type auto focus (AF) processing, automatic exposure (AE) processing, Used to perform flash pre-emission (EF) processing. Further, the image processing circuit 20 performs predetermined arithmetic processing on the data of the captured image, and also performs TTL auto white balance (AWB) processing based on the obtained arithmetic result.

Reference numeral 22 denotes a memory control circuit, which includes an A / D converter 16, a timing generation circuit 18, an image processing circuit 20, and an image display memory.
24, D / A converter 26, work memory 30, compression / expansion circuit 3
Control two. Data output from the A / D converter 16 is written to the image display memory 24 or the work memory 30 via the image processing circuit 20 or directly via the memory control circuit 22.

24 is an image display memory, 26 is a D / A converter, 28
Denotes an image display unit composed of a TFT LCD or the like. The display image data written in the image display memory 24 is sent to the image display unit 28 via the D / A converter 26, and the image is displayed. Therefore, by sequentially sending the image data to be captured to the image display unit 28, the electronic finder function is realized. The display of the image display unit 28 can be turned on / off arbitrarily according to an instruction from the system control circuit 50.When the display is turned off and the backlight is turned off, the power consumption of the electronic camera 100 is significantly reduced. Can be reduced.

Reference numeral 30 denotes a work memory, such as a semiconductor RAM, for storing photographed still images and moving images, which has a sufficient storage capacity to store a predetermined number of still images and moving images for a predetermined time. ing. Thus, even when performing continuous shooting or panoramic shooting in which a plurality of still images are continuously shot, high-speed shooting can be performed by writing a large amount of image data to the work work memory 30 at high speed. The work work memory 30 can also be used as a work area of the system control circuit 50.

Reference numeral 32 denotes a compression / decompression circuit for compressing / decompressing image data by an image compression method using a discrete wavelet transform (DWT) or the like. The written image data is written into the work memory 30 again.

An exposure control unit 40 controls the shutter 12 having an aperture function, and also has a flash dimming function in cooperation with the flash 48. Reference numeral 48 denotes a flash, which has an AF auxiliary light projection function and a flash light control function. 42 is a distance measurement control unit that controls focusing of the taking lens 10, 44 is a zoom control unit that controls zooming of the taking lens 10, 46
Is a barrier control unit that controls the operation of the barrier 102 that protects the lens 10.

As described above, the exposure control unit 40 and the distance measurement control unit 42 are controlled by the TTL method. In other words, based on the calculation result obtained by calculating the data of the captured image by the image processing circuit 20, the system control circuit 50
And the distance measurement control unit 42 is controlled.

Reference numeral 50 denotes a system control circuit for controlling the entire electronic camera 100; 52, a constant for operating the system control circuit 50;
This is a program memory for storing variables, programs, and the like.

Reference numeral 54 denotes a display unit for displaying the operating state, setting state, and various messages of the electronic camera 100 using characters, symbols, images, and the like in accordance with the execution of the program by the system control circuit 50. The display unit 54 is an electronic camera
A single unit at a position that is easy to access only near the 100 operation units, or
It is divided and arranged. Normally, the display 54 is LC
It is composed of D, LED and lamp indicators, etc., but it can also emit warning sounds, voice messages, etc. by further combining sound-producing elements. Further, some functions of the display unit 54 are arranged in the optical viewfinder 104 in an overlapping manner.

Information displayed on the LCD or the like of the display unit 54 includes, for example, single shot / continuous shooting setting, self-timer setting, compression ratio, number of recorded pixels, number of recorded images, number of remaining images, shutter speed , Aperture value, Exposure compensation setting, Flash setting, Red-eye reduction setting, Macro shooting setting, Buzzer setting, Clock battery level, Battery level, Error status, Multi-digit information, Recording media 200 The state includes the attachment / detachment state, operation of the communication interface (I / F), date and time, and connection state with an external computer.

Among the display information of the display unit 54, those displayed in the optical viewfinder 104 include, for example, a focused state, a shooting ready state, a camera shake warning, a flash charging state,
Examples include flash charging completion, shutter speed, aperture value, exposure correction status, and recording medium writing operation.

Further, information displayed on an indicator such as an LED of the display unit 54 includes, for example, an in-focus state, a preparation for photographing, a camera shake warning, a flash charging state, a flash charging completion, a recording medium writing operation, and a macro operation. There are a shooting setting notification and a secondary battery charging state.

The information displayed on the indicator such as the lamp of the display unit 54 includes, for example, a self-timer notification. This self-timer notification lamp can be shared with the AF auxiliary light source.

Reference numeral 56 denotes an electrically erasable and recordable nonvolatile memory, for example, an EEPROM or the like.

Reference numerals 60, 62, 64, 66, 68 and 70 denote input means for inputting various instructions to the system control circuit 50.
A switch, a dial, a touch panel, pointing by gaze detection, input means such as voice recognition, or the like, may be used alone or in combination.

The mode dial 60 includes a power-off mode, an automatic shooting mode, a program shooting mode, a shutter speed priority shooting mode, an aperture priority shooting mode, a manual shooting mode,
Depth of focus priority (depth) shooting mode, portrait shooting mode, landscape shooting mode, close-up shooting mode, sports shooting mode, night view shooting mode, panorama shooting mode, playback mode, multi-screen playback / erase mode, PC connection mode, etc. This is a dial for switching and setting the function mode.

The shutter switch 62 is turned on when the shutter button is pressed halfway. When the shutter switch 62 is turned on, AF processing, AE processing, AWB processing, EF processing, and the like are started.

The shutter switch 64 is turned on when the shutter button is pushed to the end. When the shutter switch 64 is turned on, the image data read from the image sensor 12 and subjected to the A / D converter 16 is transferred to the work memory via the memory control circuit 22.
Exposure processing to write in the image processing circuit 30 and the image processing circuit to the image data read from the work memory 30 by the memory control circuit 22
Development processing of performing arithmetic processing at 20, compression of image data read from the work memory 30 by the compression / decompression circuit 32,
A series of processes such as a recording process of writing the compressed data to the recording medium 200 is started.

The image display switch 66 turns on and off the display of the image display section 28. When photographing is performed using the optical viewfinder 104, power can be saved by shutting off the power supply to the image display unit 28 (LCD or backlight) by the switch 66.

The quick review switch 68 turns on and off a quick review function for automatically playing back the image just shot. Note that the electronic camera 100 of the present embodiment has a function of setting a quick review function particularly when the image display switch 66 is set to off.

Operation unit including buttons, touch panel, etc.
The 70 has a menu button, set button, macro button, multi-screen playback page break button, flash setting button, single / continuous / self-timer switching button, and menu navigation
+ (Plus) button, menu navigation-(minus) button, playback image navigation + (plus) button, playback image navigation-
(Minus) button, image quality selection button, exposure compensation button, date / time setting button, image display on / off button for setting on / off of the image display unit 28, and image display immediately after shooting Set quick review function to automatically play back the
There is an off button.

Reference numeral 80 denotes a power control unit, which includes a battery detection circuit, DC-DC
It consists of a converter, a switch circuit that switches the block to be energized, and the like.
The battery type and the remaining battery level are detected, and the DC-DC converter is controlled based on the detection result and the instruction from the system control circuit 50 to supply necessary power to each unit including the recording medium for a required period. Reference numeral 86 denotes a power supply including a primary battery such as an alkaline battery or a lithium battery, a secondary battery such as a NiCd battery, a NiMH battery, or a Li-ion battery, or a power adapter. Removably connected.

Reference numeral 90 denotes a recording medium such as a memory card or a hard disk which is detachably connected via a connector 92;
An interface (I / F) for connecting to the system bus 129 of the electronic camera 100. Reference numeral 98 denotes a recording medium attachment / detachment detection unit that detects whether a recording medium is attached to the connector 92. The recording medium attachment / detachment detecting unit 98 can also detect that, for example, a communication card described later is attached to the connector 92 instead of the recording medium.

In the present embodiment, the interface and the connector for connecting the recording medium are described as having one system. However, the interface and the connector may be two systems or three or more systems. .
In addition, PCMCIA (Per
sonal Computer Memory Card International Associati
On) card or CF (Compact Flash) card or the like may be used, and the interfaces and connectors of the two systems may have different standards.

If the interface and the connector conform to standards such as PCMCIA card and CF card, LAN card, MODEM card, USB (Universal Serial Bus) card,
IEEE1394 card, IEEE1284 card, SCSI (Small Compute
r System Interface) cards and various communication cards such as PHS communication cards can be connected, and image data and management information attached to image data can be exchanged with other computers and peripheral devices such as printers. can do.

Reference numeral 200 denotes a recording medium such as a memory card or a hard disk. The recording medium includes a recording unit 202 including a semiconductor memory and a magnetic disk, and the electronic camera 1.
Interface 204 and connector 2 for connecting to 00
06 is equipped.

Reference numeral 102 denotes a barrier that mechanically covers the imaging unit including the lens 10 to protect the imaging unit and prevent dirt from being attached or damaged.

Reference numeral 104 denotes an optical finder, which allows the user to optically observe the photographing range without using the electronic finder function of the image display unit 28. As described above, the display unit 5 is provided in the optical viewfinder 104.
Some functions of 4 are arranged.

A communication unit 110 is a serial communication function such as RS232C, USB and / or IEEE1394, and IEEE1284 and / or
Or parallel communication function such as SCSI, other, MODEM, L
It has various communication functions such as AN, wireless communication, and infrared (Ir) communication.

Reference numeral 112 is connected to the communication unit 110 and the electronic camera 10
0 is a connector that connects to other devices. Of course, the antenna corresponds to wireless communication, and the light transmitting / receiving unit corresponds to infrared communication.

[Operation of Electronic Camera] FIGS. 2 and 3 are flowcharts showing an example of a main routine of the electronic camera 100, and show a process started by the system control circuit 50 when a battery is replaced.

First, in step S101, flags, control variables, and the like are initialized, and various parts of the electronic camera are initialized. Next, in step S102, the setting position of the mode dial 60 is determined.
Move to S104. If the mode dial 60 has been set to other than power off, the process proceeds to step S103.

In step S103, the count value of an auto shutdown timer (not shown) for automatically turning off the power when the operation is not performed for a predetermined time is checked. If the count value exceeds the predetermined value, the process proceeds to step S103. Move to S104.

In step S104, an end process for turning off the power is performed. That is, the display on the display unit 54 is changed to the end state, the barrier 102 is closed to protect the imaging unit, and necessary parameters and setting values including flags and control variables and the setting mode are recorded in the nonvolatile memory 56. Then, a predetermined termination process such as shutting off unnecessary power of each unit including the image display unit 28 by the power supply control unit 80 is performed. Then, step S1
Return to 02 and wait until the mode dial 60 is set to anything other than power off.

If the count value of the auto shutdown timer has not exceeded the predetermined value, the set position of the mode dial 60 is determined again in step S105, and if the mode is set to the reproduction mode, the reproduction process is executed in step S130. After the processing, the process returns to step S102. The reproduction process reads data from the storage medium 200, decompresses the compressed data, and displays an image on the image display unit 28, but detailed description is omitted here.

If the mode dial 60 is set to the photographing mode, the flow shifts to step S106 to perform automatic exposure.
(AE), pre-flash (EF), auto white balance (AWB), and auto focus (AF) processing.

The processing in step S106 will be specifically described.
First, under the control of the system control circuit 50, image data is sequentially read into the image processing circuit 20 via the image sensor 14 and the A / D converter 16. Then, the image processing circuit 20 uses the sequentially read image data to perform TTL AE processing,
Performs calculations for F processing, AWB processing, and AF processing. Note that these processes are performed by cutting out a necessary number of necessary regions from the entire photographing region. Accordingly, in each process, an optimal calculation can be performed for each of different modes such as the center-weighted mode, the average mode, and the evaluation mode.

Based on the calculation result of the image processing circuit 20, the system control circuit 50 performs AE control using the exposure control unit 40 until the exposure is determined to be appropriate. Further, the system control circuit 50 determines whether or not a flash is necessary using the measurement data obtained by the AE control, and if necessary, sets a flash flag and charges the flash 48. When it is determined that the exposure is appropriate, the measurement data and / or the setting parameters are stored in the internal memory or the program memory 52 of the system control circuit 50.

Subsequently, based on the calculation result of the image processing circuit 20 and the measurement data obtained by the AE control, the system control circuit 50 performs the color processing used by the image processing circuit 20 until the white balance is determined to be appropriate. AW to adjust parameters
Perform B control. When it is determined that the white balance is appropriate, the measurement data and / or the setting parameters are stored in the internal memory or the program memory 52 of the system control circuit 50.

Subsequently, the system control circuit 50 uses the measurement data obtained by the AE control and the AWB control to perform the AF control using the distance measurement control unit 42 until it is determined that the object is in focus. When it is determined that the object is in focus, the measurement data and / or
Alternatively, the set parameters are stored in the internal memory or the program memory 52 of the system control circuit 50, and the distance measurement and the photometry processing are completed.

Next, in step S107, the image processing circuit 20 performs image processing for converting the captured image data into image data (display data) suitable for displaying on the image display unit 28. Specifically, processing such as color conversion, white balance adjustment, and luminance adjustment is performed on the captured image data to generate display data. The display data is stored in step S108
Then, the image data is stored in the image display memory 24 by the memory control circuit 22.

Next, in step S109, it is determined whether or not the guidance photographing mode is set. If not, the display data stored in the image display memory 24 is converted to D / A in step S111. The image is converted and displayed on the image display unit 28. On the other hand, if the guidance shooting mode has been set, in step S110, the memory control
The guidance frame information is read from, for example, step S111.
Then, an image in which the guidance frame is superimposed on the image of the display data stored in the image display memory 24 is displayed on the image display unit 28.

In the present embodiment, as an example, a guidance frame is displayed when portrait photography is performed, and a guidance frame (ROI frame) for portrait photography as shown in FIG. It shall be displayed at 28.

Next, in step S112, the shutter switch
It is determined whether or not SW1 of 66 is on, and if SW1 is off, the process returns to step S102. If SW1 is on, the process proceeds to step S113, where the optimum exposure amount is determined from the photometric value at that time, and the exposure control unit 40 is controlled. Subsequently, step S114
Then, the lens drive amount is determined from the distance measurement value at that time, and the distance measurement control unit 42 is driven.

Next, in step S115, the shutter switch
It is determined whether or not SW2 of 66 is on, and if SW2 is off, the process returns to step S102. If SW2 is on, the process proceeds to step S116 to start a shooting operation.

In step S116, similarly to step S109, it is determined whether or not the guidance photographing mode is set. If the guidance shooting mode has been set, the guidance frame is set to ROI in step S117, and then shooting is performed in step S118. System control circuit
50 controls the exposure control unit 40 in accordance with the photometric data stored in the internal memory or the memory 30 to set the aperture of the shutter 12 and open the shutter 12, and then sets the exposure time based on the photometric data. After the elapse, the shutter 12 is closed. Then, the charge signal is read out from the image sensor 14, and A /
Through the D converter 16, the image processing circuit 20, the memory control circuit 22, or without the image processing circuit 20, the memory
The photographed image data is written in 30.

In step S119, the image processing circuit 20 generates a luminance signal and a color difference signal from the photographed image data stored in the memory 30, and stores the luminance signal and the color difference signal in the memory 30 again. Further, the system control circuit 50 causes the image data to be read from the memory 30 via the memory control circuit 22 and causes the image display memory 24 to transfer the image data for display.

Next, in step S120, the system control circuit
50 compresses and encodes the luminance signal and the chrominance signal stored in the memory 30 and stores them in the memory 30. However, when the ROI is specified, compression encoding is performed in consideration of the specified portion. JPEG200 for compression coding considering ROI
Various methods proposed and studied with 0 or the like may be used. For example, the method is performed as follows.

First, an ROI in an image is created as a bit plane image, and this is used as an ROI mask. DWT image
When encoding the coefficients, if a certain coefficient is a coefficient of an image portion belonging to the ROI, a code (ROIMASK code) indicating that the coefficient belongs to the ROI is added to the coefficient. And ROI part and non
The ROI part is compression-encoded as a separate sequence (code sequence). In other words, it seems that there is an ROI image and a non-ROI image from the decoder that expands the compression-encoded data.

The following method may be used. As before, RO
When an I mask is generated and a coefficient obtained by DWT of an image is encoded, if a certain coefficient is a coefficient of an image portion belonging to the ROI, the coefficient is shifted up by a predetermined level. That is, JPEG
In 2000, the upper bit plane is preferentially encoded at the time of encoding, and the lower bit plane is omitted depending on the situation.
Therefore, the ROI portion, which is the shifted-up portion, is coded to relatively lower bits, and is compression-coded with high image quality.

Of course, the present embodiment relates to the specification of the ROI, and the ROI may be encoded and decoded by a method other than the above, and the encoding and decoding of the ROI is not limited.

The data compressed and encoded as described above is stored in the memory 30, converted into a data file of a predetermined format in step S121, and transferred to the recording medium 200 via the interface 90 and the connector 92. It is memorized. Thereafter, the process proceeds to step S1 in preparation for the next operation.
Return to 02.

FIG. 5 is a diagram showing an example of the format of compressed data in the present embodiment. In FIG. 5, reference numeral 403 denotes compressed image data. Header information 401 and 402 are recorded before the compressed image data 403. In the image property 401, various property information such as the vertical and horizontal sizes of the image and the model name of the camera used for shooting are described. The ROI property 402 is recorded when the compressed data 403 has an ROI, and describes what the ROI of the compressed data 403 has. The ROI property 402 is, for example,
This is one byte of information, and the following relationship exists between each value and the meaning of the ROI. Definition of ROI 0: Main subject area arbitrarily specified by user 1: Area in guidance frame shot in guidance shooting mode 2: Subject area when shot in template composite shooting mode Note that the subject area is a template Area other than image

Assuming that the ROI is specified in the guidance shooting mode, the ROI property 4
“1” is stored in 02. Note that in the format shown in FIG. 5, the ROI information is expressed so as to be included in the compressed data 403, but the present invention is not limited to this, and a format in which the ROI information is included in the header portion may be used.

As described above, according to the first embodiment, when shooting is performed in the guidance shooting mode, the inside of the guidance frame is
Since it is automatically set as the ROI, the photographer who is the user of the electronic camera can save the trouble of setting the ROI every time shooting is performed.

[0076]

Second Embodiment Hereinafter, an image processing apparatus according to a second embodiment of the present invention will be described. Note that, in the present embodiment,
Components that are substantially the same as those in the first embodiment are given the same reference numerals, and detailed descriptions thereof are omitted. In the second embodiment, the ROI is automatically set in the template composite shooting mode.

FIGS. 6 and 7 show an electronic camera 1 according to the second embodiment.
This is a flowchart showing an example of the main routine of 00, which is almost the same as the processing of the first embodiment shown in FIGS. 2 and 3.
However, the processing shown in FIGS. 6 and 7 differs from the processing corresponding to steps S109, S110, S116, S117 and S121 in FIGS. 2 and 3. The processing of S109 ', S110', S116 ', S117', and S121 ', which are denoted by reference numerals, will be described below.

In step S109 ', it is determined whether or not the template image composite photographing mode is set. If not, in step S111, the image display memory 24 is set.
D / A conversion of the display data stored in the
To be displayed. On the other hand, if the template composite shooting mode has been set, in step S110 ′, the memory control circuit 2
The template image data is read from the memory 30 or the like by 2 and the image in which the template image is combined with the image of the display data stored in the image display memory 24 is displayed on the image display unit 28 in step S111.

The template image is rectangular, and its image data has three planes of red (R), green (G) and blue (B), and can have values of 0 to 255 for each color.
The invalid area (subject area) of the template image data is
R = 0, G = 0, and B = 0 are set as values indicating the invalid area. Therefore, the process of synthesizing the template image is
The image data is written to the image display memory 24 for the pixels other than the invalid value.

Next, in step S116 ', step S109'
As in the case of, it is determined whether or not the template image composite shooting mode is set. If the template image composite shooting mode is set, the subject area is set to the ROI in step S117 ′, and then shooting is performed in step S118 ′.

Next, in step S121 ', substantially the same processing as in the first embodiment is performed, but the format of the compression-encoded image data is as shown in FIG. Information 401, 402 and 501 are recorded. The template ID 501 is an ID for uniquely recognizing the template image used at the time of template synthesis shooting.
It is. Note that the value “2” is stored in the ROI property 402 as described above.

As described above, according to the second embodiment, when photographing is performed in the template image composite photographing mode, the subject area is automatically set as the ROI, so that the photographer who is the user of the electronic camera can perform photographing. This eliminates the need to set the ROI each time.

[0083]

Third Embodiment As a third embodiment, an example will be described in which ROIs are color-coded according to the ROI property 402 performed in step S130 (reproduction processing) of the first and second embodiments, and markers are displayed. FIG. 9 is a flowchart showing the reproduction process, which is a process executed by the system control unit 50.

In step S601, the compressed data file stored in the recording unit 202 in the format shown in FIG. Of course, if the compressed data file of the image to be reproduced has already been read into the memory 30, step S601 is not necessary.

Next, in step S602, the compressed data stored in the memory 30 is expanded by the compression / expansion unit 32, and the expanded image data is stored in the image display memory 24. In the case of compressed data having an ROI, ROI information obtained at the time of decompression is written to the memory 30. The following description assumes that there is ROI information. If the compressed data of the image to be reproduced has already been decompressed and stored in the image display memory 24, step S602 is not necessary.

Next, in step S603, the ROI is
The information is read, and the pixel value of the image display memory 24 corresponding to the frame position of the ROI is overwritten according to the following rules. That is,
Referring to the value of the ROI property 402, for example, if the ROI property is “0”, the pixel value of (R, G, B) = (255, 0, 0)
If “1”, the pixel value of (R, G, B) = (0, 255, 0) is written, and if “2”, the pixel value of (R, G, B) = (0, 0, 255) is written.

Next, in step S604, the image display memory 24
Is converted into an analog signal by the D / A converter 26 and supplied to the image display unit 28, and the reproduction process is completed. The image display unit 28 displays an image in which an ROI frame as shown in FIG. 4 is displayed. The displayed ROI frame is red if the main subject area arbitrarily specified by the user (ROI property is “0”), and if the area is within the guidance frame shot in the guidance shooting mode (“1”). It is displayed in green if it is green, and if it is a subject area ("2" in the same case) when it is shot in the template composite shooting mode.

As described above, the ROI frame is displayed when the image is reproduced, and the ROI frame is displayed in different colors according to the properties of the ROI, so that the user can recognize the ROI and its properties at a glance.

[0089]

[Fourth Embodiment] As a fourth embodiment, step S119 of the first and second embodiments (processing of generating luminance and color difference signals)
An example in which image processing is performed on an area other than the ROI in accordance with the ROI property 402 between (Step S131) and Step S120 (compression encoding processing) will be described. FIG. 10 is a flowchart in which image processing for non-ROI is added, and is a process executed by the system control unit 50.

That is, in step S131, the image data and the ROI information stored in the memory 30 are read by the image processing unit 20, the image processing is performed on the non-ROI, and the result is written into the memory 30 again. As image processing for non-ROI,
For example, there is an example in which a cross filter as shown in FIG. 11 is applied.

The selective application of image processing to the inside or outside of the designated area is generally performed by photo retouching software or the like. The same processing and procedure may be used. Therefore, a personal computer running an image processing program such as photo retouching software reads a compressed data file in a format as shown in FIG. 5 or 8 from the recording medium 200 or the like, and expands the compressed data to obtain image data. If the non-ROI is subjected to image processing and the image data subjected to image processing is compressed and stored in the recording medium 200 or the like, the same result as described above can be obtained.

[0092] Examples of preferable image processing to be applied to the non-ROI include a cross filter, a low-pass filter, a color conversion to a sepia tone, a fill with a user-specified color, posterization, and addition of noise.

With the above configuration, image processing or the like for giving a special effect to the ROI can be performed. Examples of image processing that gives a special effect include, for example, enhancement,
Examples include a high-pass filter, noise removal, and color moiré elimination.

Further, according to the above configuration, it is also possible to determine the area to be subjected to image processing and the processing content according to the ROI property 402. For example, the following shows an example of the relationship between image processing performed in response to the ROI property and an area.
It is not limited to this. ROI property: 0

(Main subject area arbitrarily specified by user) Processing area: ROI Processing content: Eliminating color moiré ROI property: 1 (Area in guidance frame shot in guidance shooting mode Processing area: Non-ROI Processing content: Cross Filter ROI property: 2 (subject area when shot in template image synthesis mode) Processing area: Non-ROI Processing content: synthesis of template image In this way, image processing is performed selectively outside the ROI to achieve ROI image quality It is possible to easily obtain an artistic photographing result that preserves the image. In addition, it is possible to automatically determine which image processing should be applied to which area according to the ROI property, Even a user who is unfamiliar with the image processing to give can easily obtain an image with an appropriate special effect.

[0096]

[Other Embodiments] Even if the present invention is applied to a system including a plurality of devices (for example, a host computer, an interface device, a reader, a printer, etc.), an apparatus (for example, a copying machine) Machine, facsimile machine, etc.).

Further, an object of the present invention is to supply a storage medium (or a recording medium) on which a program code of software for realizing the functions of the above-described embodiments is recorded to a system or an apparatus, and to provide a computer (a computer) of the system or the apparatus. It is needless to say that the present invention can also be achieved by a CPU or an MPU) reading and executing the program code stored in the storage medium. In this case, the program code itself read from the storage medium implements the functions of the above-described embodiment, and the storage medium storing the program code constitutes the present invention. Also,
When the computer executes the readout program code, not only the functions of the above-described embodiments are realized, but also the operating system (OS) running on the computer based on the instructions of the program code.
It is needless to say that a case in which the functions of the above-described embodiments are implemented by performing part or all of the actual processing.

Further, after the program code read from the storage medium is written into a memory provided in a function expansion card inserted into the computer or a function expansion unit connected to the computer, the program code is read based on the instruction of the program code. Needless to say, the CPU included in the function expansion card or the function expansion unit performs part or all of the actual processing, and the processing realizes the functions of the above-described embodiments.

When the present invention is applied to the storage medium, the storage medium has program codes corresponding to the flowcharts shown in FIGS. 2 to 3, 6 and 7, and / or 11 described above. Will be stored.

[0100]

As described above, according to the present invention,
It is possible to provide an image processing apparatus and a method thereof that can easily set a region of interest in an image to be captured.

Further, it is possible to provide an image processing apparatus and an image processing method for performing image processing in accordance with a region of interest set in an image.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a configuration example of an electronic camera according to a first embodiment;

FIG. 2 is a flowchart showing an example of a main routine of the electronic camera;

FIG. 3 is a flowchart showing an example of a main routine of the electronic camera;

FIG. 4 is a diagram showing a display example of a guidance frame for portrait shooting;

FIG. 5 is a diagram showing a format example of compressed data;

FIG. 6 is a flowchart illustrating an example of a main routine of the electronic camera according to the second embodiment;

FIG. 7 is a flowchart illustrating an example of a main routine of the electronic camera according to the second embodiment;

FIG. 8 is a diagram showing a format example of compressed data;

FIG. 9 is a flowchart showing a reproduction process;

FIG. 10 is a flowchart in which image processing for non-ROI is added,

FIG. 11 is a diagram illustrating an example of image processing using a cross filter.

Claims (15)

[Claims] 1. An image processing apparatus comprising: an image capturing unit that captures an image; a display unit that displays an image of a subject; a processing unit that processes a captured image; and a control unit that controls them. And displaying on the display means a frame that guides shooting on the image of the subject, and causing the processing means to process the area inside the frame of the captured image under a condition different from the area outside the frame. Characteristic image processing device. 2. An image processing apparatus comprising: an image capturing unit for capturing an image; a display unit for displaying an image of a subject; a processing unit for processing the captured image; and a control unit for controlling the same. And displaying the template image on the image of the subject on the display unit, and causing the processing unit to process a region other than the template image region of the photographed image under a condition different from the template image region. Image processing apparatus. 3. The image processing apparatus according to claim 1, wherein the processing unit compresses the image according to a JPEG2000 system. 4. A memory control unit for storing a data file of an image processed by the processing unit in a memory, wherein the data file includes information indicating the frame or the template image area. 4. The image processing device according to claim 1, wherein: 5. An image processing apparatus comprising: reading means for reading a data file stored in a memory; reproducing means for reproducing an image from a read data fail; display means for displaying an image; and control means for controlling them. An image processing apparatus, wherein, when the data file includes information indicating a region of interest of the image, the control unit causes the display unit to display an image indicating the region of interest over a reproduced image. . 6. The apparatus according to claim 5, wherein when a property is set in the attention area, the control unit sets a color corresponding to the property in an image indicating the attention area. Image processing device. 7. An image processing apparatus comprising: reading means for reading a data file stored in a memory; reproducing means for reproducing an image from a read data fail; display means for displaying an image; and control means for controlling them. An image processing apparatus, wherein when the data file includes information indicating an attention area of an image, the control means selectively performs predetermined image processing on the attention area or outside the area. 8. A display means for displaying an image of a subject displays a frame for guiding the shooting on the image of the subject so as to be superimposed on the image of the subject, and the area inside the frame of the image shot by the imaging means is defined as the area outside the frame. An image processing method characterized by processing under different conditions. 9. A display unit for displaying an image of a subject, a template image is superimposed on the image of the subject and displayed, and an area outside the template image area of an image photographed by an imaging unit is different from the template image area. An image processing method characterized by processing under conditions. 10. A data file stored in a memory is read, and an image is reproduced from the read data file. If the data file includes information indicating a region of interest of the image, the region of interest is superimposed on the reproduced image. An image processing method characterized by displaying an image indicating the following. 11. A data file stored in a memory is read, and an image is reproduced from the read data fail. If the data file includes information indicating a region of interest of the image, the data file is selected outside the region of interest or outside the region. An image processing method characterized by performing predetermined image processing. 12. A recording medium on which a program code for image processing is recorded, wherein said program code causes at least a display means for displaying an image of a subject to superimpose a frame for guiding photographing on the image of the subject. A recording medium comprising: a step code; and a step code for processing an area inside the frame of an image captured by an imaging unit under conditions different from those of the area outside the frame. 13. A recording medium on which a program code for image processing is recorded, wherein said program code is a code for a step of displaying a template image superimposed on an image of a subject on at least display means for displaying the image of the subject. And a code for processing an area outside the template image area of an image captured by an imaging unit under a condition different from that of the template image area. 14. A recording medium on which a program code for image processing is recorded, wherein the program code includes at least a code for reading a data file stored in a memory, and reproduces an image from the read data file. A recording medium, comprising: a step code; and a code for displaying an image indicating the region of interest superimposed on a reproduced image when information indicating the region of interest of the image is included in the data file. 15. A recording medium on which a program code for image processing is recorded, wherein the program code reproduces an image from at least a code for reading a data file stored in a memory and a read data fail. A recording medium comprising: a step code; and a code for selectively performing a predetermined image processing on or outside the region of interest when the data file includes information indicating the region of interest of the image. .