JP2010510717A - System and method for performing camera photometry based on skin color detection - Google Patents

Abstract

A method of adjusting exposure settings for a digital camera includes searching at least a portion of the field of view of the camera for pixels having a flesh tone. The flesh tone pixels are analyzed to identify at least one flesh tone spot. Spot metering of the field of view of the camera is carried out such that at least one flesh tone spot is given greater weight than the rest of the field of view. Then, the exposure settings are adjusted based on the spot metering.

Description

  The present invention relates to photographic technology, and more particularly to a system and method for performing camera photometry based on flesh tone detection.

  The exposure setting value of the digital camera can be set based on spot metering. A typical spot photometric technique involves monitoring the illumination level across some portion of the camera field of view or across the camera field of view. The illumination levels are integrated together and the lighting conditions are evaluated and exposure settings such as flash intensity, aperture value (f-stop) and shutter speed are adjusted (if a flash is used). In the integration of the illumination level, a high weight is given to a certain field of view of the camera. For example, the camera user gives a higher weight to the central square section of the scene than the rest of the field of view, under the prediction that the desired target of the photograph tends to be centered in the camera's field of view.

  There are several situations where spot metering is not effective. For example, if the user intends to take a portrait photo and there is a dark theme in the photometric spot (such as the dark branches of a tree behind the person posing for the photo), the camera May sense that the photo is of a dark scene and set the exposure setting accordingly. As a result, the human face may be overexposed. Similarly, when a bright theme exists in the photometric spot, the face may be underexposed. In both of the above situations, it is possible to image relatively unimportant details (such as in the case of the above-mentioned branches), but the resulting photographic face is too bright (“fading” Can be washed-out) or too dark. There is a similar problem in area metering.

  In addition, most digital cameras employ analog gain adjustment to “fit” the dynamic range of an analog / digital converter (ADC) into the imaging scene. Many ADCs are limited in their range (eg, about 8 bit words) for cost, sensitivity, size and speed reasons. Face details tend to be dynamically compressed in the range of the ADC when photometry results in a face image as a dark or bright area. In particular, in spot metering, area metering, and combinations thereof, it is often impossible to center the face dynamically within the range of the ADC. This is because when taking a picture of one or more persons, one or more faces may not fit within the spot used for spot metering, or one or more faces are effective for area metering. May not have an area large enough to affect the camera, or the camera may select the wrong metering mode (even if applicable for the camera) Because. Therefore, the photometry may fail to adjust the exposure setting value for high quality imaging of a person present in the camera's field of view. Sometimes post-processing may improve some of the dynamically compressed photos, but once the data is lost due to compression, it is usually impossible to restore sharpness and post-processing. Is known to produce noise.

  Some cameras allow manual adjustment of exposure settings to optimize facial details. However, most camera users are not familiar with what actions should be taken to manually adjust camera settings.

  In order to improve photographic quality, there is a need for technology relating to an automated system and method for camera photometry that leads to improved image quality of the person appearing in the photograph. According to one aspect of the present invention, camera photometry is performed using skin color detection, whereby a face or a person appearing in a photograph has high definition and sharpness.

  According to one aspect of the present invention, a method for adjusting an exposure setting value of a digital camera includes a step of searching for a skin color pixel from at least a part of a field of view of the camera, and analyzing the skin color pixel to obtain at least one skin color spot. Performing a spot metering of the camera field of view by assigning a higher weight to the at least one skin tone spot than the rest of the field of view, and adjusting the exposure setting based on the spot metering A step of performing.

  According to one embodiment of the method, the analysis of the flesh color pixels includes a grouping of flesh color pixels by relative approximation between the pixels.

  According to one embodiment of the method, the flesh color spot region has at least a threshold proportion of flesh color pixels.

  According to one embodiment, the method further comprises the step of reducing the flesh spot size before performing spot metering.

  According to one embodiment, the method further comprises the step of converting the RGB output of the sensor that images the field of view into coordinate-based color values for the individual pixels.

  According to one embodiment of the method, the skin tone is defined by a predetermined portion of a coordinate-based color space.

  According to one embodiment of the method, the predetermined portion of the coordinate-based color space is elliptical.

  According to one embodiment of the method, the elliptical portion of the coordinate-based color space is approximated by a set of squares.

  According to one embodiment, the method further comprises iteratively adjusting the exposure setting from dark to bright until skin color saturation is detected.

  According to one embodiment, the method further comprises performing at least a portion of the iterative adjustment in the reverse direction upon detecting skin color saturation.

  According to another aspect of the present invention, the camera images the field of view and outputs a signal indicating luminance and color for a plurality of pixels, a step of identifying the skin color pixel, and the skin color pixel is analyzed. Identifying at least one flesh-colored spot; giving at least one flesh-colored spot a higher weight than the rest of the field of view and performing spot metering of the camera field of view; an exposure setting based on the spot metering And a controller for adjusting an exposure setting value of the camera by executing the step of adjusting the camera.

  According to one embodiment of the camera, the controller executes a logic operation instruction that adjusts an exposure setting value.

  According to one embodiment of the camera, the analysis of the flesh color pixels by the controller includes a grouping of flesh color pixels by relative approximation between the pixels.

  According to one embodiment of the camera, the skin color spot region has at least a threshold ratio of skin color pixels.

  According to one embodiment of the camera, the controller reduces the size of the flesh spot before performing spot metering.

  According to one embodiment of the camera, the sensor output value includes a set of RGB values for individual pixels, and the controller converts the sensor RGB output values to coordinate-based color values for the individual pixels.

  According to one embodiment of the camera, the skin color is defined by a predetermined portion of a coordinate-based color space.

  According to one embodiment of the camera, the controller iteratively adjusts the exposure setting value from dark to bright until skin color saturation is detected.

  According to one embodiment of the camera, when the controller detects skin color saturation, it performs at least a portion of the iterative adjustment in the reverse direction.

  According to one embodiment of the camera, the camera forms part of a mobile radio terminal with a radio circuit for establishing a call over a communication network.

  These and further features of the present invention will be apparent with reference to the following description and attached drawings. In the following description and drawings, specific embodiments of the present invention have been disclosed in detail to illustrate some of the ways in which the principles of the present invention can be employed. It should be understood that the scope of the present invention is not limited to these embodiments. The present invention includes all changes, modifications, and equivalents included in the description and the gist of the claims.

  The features described and / or illustrated with respect to one embodiment may be in one or more other embodiments, in combination with features of other embodiments, or in place of features of other embodiments. It can be used in the same or similar manner, at least in any form.

  As used herein, the terms “comprises” and “comprising” are used to identify the presence of the mentioned feature, integer, step or configuration, but these It is emphasized that it does not exclude the presence or addition of one or more other features, integers, steps, configurations, or groups thereof.

1 is a schematic front view of a mobile telephone provided with a camera assembly as an exemplary electronic device according to an embodiment of the present invention. FIG. 2 is a schematic rear view of the mobile phone of FIG. 1. It is a schematic block diagram which shows the operation | movement part of the mobile telephone of FIG. FIG. 2 is a schematic diagram of a communication system in which the mobile telephone of FIG. 1 can operate. It is a flowchart showing an example of the camera photometry method concerning the present invention. 6 is a flowchart showing skin color pixel identification and skin color pixel analysis of the method of FIG. 5.

  Embodiments of the present invention will now be described with reference to the drawings, in which like reference numerals are used to refer to like elements throughout. It should be understood that these figures are not necessarily to scale.

  One aspect of the present invention relates to photographic technology. The technology described in this specification can be applied to photography using a digital still camera and video photography using a digital video camera. It will be understood that a digital camera may be provided with a function for capturing both still images and moving images. If the camera is equipped with a suitable sensor for detecting skin color as described in this specification, the technique described in this specification can also be used for a film camera.

  The techniques described herein can be performed by any type of electronic device including a camera. For example, a dedicated still digital camera and / or video digital camera can be configured in accordance with the present invention.

  Many mobile telephones include a camera that can be configured in accordance with the present invention. For example, the following description is for a mobile phone that includes a camera assembly. However, it will be appreciated that the invention is not intended to be limited to mobile telephones and that the invention may relate to any type of suitable electronic equipment. For example, a dedicated camera, a media player including a camera, a game device including a camera, a computer including a camera, and the like can be included in the present invention. For convenience of description herein, the interchangeable terms “electronic equipment” and “electronic device” include portable wireless communication equipment. A “portable wireless communication device” will be referred to hereinafter as a “mobile wireless terminal”, which includes a mobile phone, a pager, a communicator, an electronic notebook, a personal digital assistant (PDA). All devices such as smartphones, portable communication devices, etc. are included.

  Referring initially to FIGS. 1-3, an electronic device 10 is shown. The electronic device 10 includes a camera assembly 12 that is configured to perform photometry and adjust exposure settings based on skin color presented in the field of view of the camera. Additional details and operation of the camera assembly 12 are described in further detail below. The photometric technique described in this specification can be realized as an executable program that is resident in the electronic device 10 and executed by the electronic device 10. In one embodiment, camera assembly 12 and / or electronic device 10 may include a controller or processor that executes a program stored on a computer or machine-readable medium. This program may be a stand-alone software application, or may form part of a software application that performs additional tasks related to the electronic device 10.

  The electronic device of the illustrated embodiment is a mobile telephone and will be referred to as mobile telephone 10 below. The mobile phone 10 is shown as having a “brick” or “block” type housing, but other types of shell type housings (such as “flip” open type housings) or sliding type housings. A housing may be used.

  The mobile telephone 10 includes a display 14. The display 14 displays information such as operating status, time, telephone number, contact information, various navigation menus, etc. to the user. With this information, the user can use various features of the mobile telephone 10. It is also possible to visually display the contents received by the mobile telephone 10 and the contents taken out from the memory 16 (FIG. 3) of the mobile telephone 10 using the display 14. The display 14 can be used to present to the user graphic images such as images, video footage, photos, mobile television content, game related video images. The display 14 can also be used as an electronic viewfinder for the camera assembly 12.

  The keypad 18 enables an input operation by the user. For example, keypad 18 typically includes alphanumeric keys that allow entry of alphanumeric information such as telephone numbers, telephone lists, contact information, notes, and the like. In addition, the keypad 18 typically has special function keys, such as a “send call” key, a “end call” key, or a “stop” key for calling to initiate or answer a call. include. Special function keys may include menu navigation keys that facilitate navigation through menus displayed on the display 14 and selection keys. Special function keys may include audiovisual content playback keys for starting, stopping and pausing playback, skipping or repeating tracks, and the like. Other keys associated with the mobile phone may include volume keys, audio mute keys, power on / off keys, web browser start keys, and the like. Some or all of the keys may be used as soft keys associated with the display. It is also possible to implement a key or key-like function as a touch screen associated with the display 14. When the camera assembly 12 is activated, the operation of the camera assembly 12 can be controlled using keys from the keypad 18. For example, one of these keys can function as a shutter key, and the other key can control the zoom of the camera assembly.

  The mobile telephone 10 allows the mobile telephone 10 to establish a call and / or exchange signals with a called / calling device that is usually another mobile telephone or a wired telephone A calling circuit is included. However, the called / calling device need not be another telephone, and may be any other device such as an Internet web server, a content providing server, or the like. The call can take any suitable form. For example, the call may be a normal call established via a cellular circuit switched network, or via a packet switched function of the cellular network, or other packet switched such as WiFi, WiMAX, etc. It may be a voice over internet protocol (VoIP) phone established over a network. Another example is a video phone established over a cellular network or other network.

  The mobile phone 10 is a text message, e-mail message (representing a simple message service, sometimes referred to as “SMS”), or a message (representing a multimedia message service, sometimes referred to as “MMS”). It can be configured to send, receive and process data such as multimedia messages, image files, video files, audio files, ringtones, streaming audio, streaming video, data feeds (including podcasts), and the like. Such data processing steps include storing data in memory 16 to allow user interaction with the data, executing an application, displaying video and image content associated with the data, Outputting audio sound associated with the data.

  FIG. 3 is a functional block diagram of the mobile telephone 10. For the sake of brevity, generally conventional features of the mobile telephone 10 will not be described in detail herein. The mobile telephone 10 includes a control circuit 20 that is configured to perform control over the functions and operations of the mobile telephone 10. The control circuit 20 may include a processing device 22 such as a CPU, a microcontroller, or a microprocessor. The processing device 22 is intended to perform the operation of the mobile telephone 10 by executing program code stored in a memory (not shown) in the control circuit 20 or in an independent memory such as the memory 16. To do. The memory 16 can be realized by a suitable device such as a buffer memory, a flash memory, a hard disk drive, a removable medium, a volatile memory, and a nonvolatile memory.

  With continued reference to FIGS. 1-3, the mobile telephone 10 includes an antenna 24 coupled to a radio circuit 26. As is commonly done, the radio circuit 26 includes a radio frequency transceiver for transmitting and receiving signals via the antenna 24. The radio circuit 26 can be configured to operate in a mobile communication system, and can be used to transmit and receive data and / or audiovisual content. The receiver for performing the interaction between the mobile radio network and at least one of the broadcast networks includes GSM, CDMA, WCDMA, GPRS, MBMS, WiFi, WiFi, DVB-H, ISDB-T, etc. It includes, but is not limited to, higher versions of the standard.

  The mobile telephone 10 further includes an audio processing circuit 28 for processing the audio signal transmitted by the radio circuit 26 and the audio signal received from the radio circuit 26. As is generally done, a speaker 30 and a microphone 32 that can be listened to and communicated with via a mobile telephone 10 are coupled to an audio processing circuit 28. The radio circuit 26 and the audio processing circuit 28 are individually coupled to the control circuit 20 with the intention of performing the entire process. Audio data can be passed from the control circuit 20 to the audio processing circuit 28 for playback to the user. The audio data is stored in the memory 16 and further includes audio data obtained from an audio file extracted by the control circuit 20, or audio data received in the form of streaming audio data from a mobile radio service, or the like. There may be. The audio processing circuit 28 includes any suitable buffer, decoder, amplifier, and the like.

  The display 14 can be coupled to the control circuit 20 by a video processing circuit 34 that converts video data used to drive the display 14 into a video signal. Video processing circuit 34 includes any suitable buffer, decoder, video data processor, and the like. The video data is derived from the video file generated by the control circuit 20 and stored in the memory 16 and derived from the incoming video data stream received by the wireless circuit 28 or obtained by other suitable methods. May be.

  The mobile telephone 10 further includes one or more I / O interfaces 36. The I / O interface 36 may be a typical mobile telephone I / O interface and may include one or more electrical connectors. As is commonly done, the I / O interface 36 may be used to couple the mobile phone 10 that charges the battery of the power supply unit (PSU) 38 within the mobile phone 10 to a battery charger. Additionally or alternatively, the I / O interface 36 serves to connect the mobile telephone 10 having a wired interface with the mobile telephone 10 to a headset assembly such as a personal hands-free (PHF) device. is there. Further, the I / O interface 36 serves to connect the mobile telephone 10 to a personal computer or other device via a data cable for exchanging data. The mobile telephone 10 can receive operating power via the I / O interface 36 when connected to a vehicle power adapter or an electrical outlet power adapter.

  The mobile telephone 10 also includes a timer 40 that performs a timing function. Thereby, it is possible to measure the duration of the call, generate a time stamp or a date stamp, and the like. The mobile telephone 10 also includes a position data receiving device 42 such as a GPS receiver or a Galileo satellite system receiver.

  The mobile telephone 10 also includes a wireless interface 44 for establishing communications with accessories, other mobile wireless terminals, computers or other devices with a local wireless interface 44 such as an infrared transceiver or an RF adapter (such as a Bluetooth adapter). . For example, in embodiments where the headset assembly includes a corresponding wireless interface, the local wireless interface 44 can operably couple the mobile telephone 10 with a headset assembly (such as a PHF device).

  Still referring to FIG. 4, the mobile telephone 10 is configured to operate as part of the communication system 46. The communication system 46 manages a call sent by the mobile telephone 10 and directed to the mobile telephone 10, transmits data to the mobile telephone 10, and performs a server (or any other support function). Communication network 48 having a plurality of servers) 50. Server 50 communicates with mobile telephone 10 via a transmission medium. The transmission medium may be any suitable device or assembly including, for example, a communication tower (such as a cell tower), other mobile phones, wireless access points, satellites, and the like. A part of the network may include a wireless transmission path. Network 48 may support communication activities of multiple mobile telephones 10 and other types of end user devices. The server 50 can be configured as a general computer system used to execute a server function. For example, the server 50 is configured to execute software including logical operation instructions that embody the function of the server 50. A processor and a memory for storing such software are included.

  With continued reference to FIGS. 1-3, the camera assembly 12 includes a controller 52, such as a digital signal processor (DSP). In other embodiments, some or all of the control functions of the camera assembly 12 described as being performed by the controller 52 can be performed by the control circuit 20, and the controller 52 can be miniaturized or omitted. Is possible. The controller 52 controls various aspects of the operation of the camera assembly 12 including, but not limited to, photometric processing and exposure setting values. The controller 52 can coordinate storage of image data and video data captured by the camera assembly 12. For example, the memory 16 can store image data or video data in a corresponding image file or video file.

  The controller 52 can execute a program to implement the control function of the camera assembly 12. Methods for programming a camera or mobile phone to operate the functions described herein and to operate and execute logic functions in conjunction with the control of the camera assembly 12 include computer programming, in particular, camera, mobile Those skilled in the art of application programming for telephones and other electronic devices will be apparent. Therefore, details regarding specific programming code are omitted. In addition to executing such program code by the controller 52 in accordance with a preferred embodiment of the present invention, dedicated hardware, firmware, software, or combinations thereof as described above without departing from the scope of the present invention. It is also possible to perform functions.

  In the illustrated embodiment, the camera assembly 12 is a digital camera using a sensor 54 that images the field of view defined by the imaging optics 56 of the camera assembly. The exposure meter 58 detects the illumination state across the field of view. Flash 60 provides auxiliary lighting during photography.

  With further reference to FIGS. 5 and 6, there is shown an example of logical processing that implements a method for performing camera photometry based on skin color detection. This example method can be executed by executing the program code using the controller 52, for example. Accordingly, the flowcharts of FIGS. 5 and 6 can be considered to depict the method performed by the camera assembly 12. 5 and 6 show a specific order in which the functional logic blocks are executed, the order of execution instructions of the blocks can be changed in accordance with the order shown. Further, two or more blocks shown in succession may be executed simultaneously or by partial matching. Certain blocks can be omitted. In addition, any number of functions, logic processes, commands, state variables, semaphores, or messages can be added to the process for enhanced utility, billing, performance, measurement, troubleshooting, etc. All such modifications are to be understood as falling within the scope of the present invention.

  In block 62 for identifying skin color pixels, processing for a method of performing camera photometry based on skin color detection is started. A detailed approach for identifying skin color pixels is shown in FIG. The exemplary approach sequentially determines whether individual pixels contain flesh tones, but as can be appreciated, this determination can be made for multiple pixels simultaneously. In block 64 where a set of pixel values is received, a process for determining skin color pixels is initiated. For example, the sensor 54 outputs a set of red, green and blue (RGB) values corresponding to the pixels input to the controller 52.

  At block 66, the RGB value set corresponding to the pixel is converted to a CIE color chart (or color space) value. For example, an RGB value set can be converted to a YUV value set. The YUV value is a value related to the coordinate system of the color chart, the Y value represents luminance (luminance), and the U and V values represent chrominance (color) components or points on an XY style color plane. The upper right quadrant of the plane (a positive value for both U and V) generally contains purple, the lower right quadrant of the plane (a positive value for U and a negative value for V) generally contains blue, and the upper left of the plane Quadrants (negative U values and positive V values) typically include red, orange and yellow, and the lower left quadrant of the plane (negative values for both U and V) generally includes green. However, these colors tend to mix together across the ends of the plane. The conversion from RGB to YUV and the conversion from YUV to the original RGB can be performed using linear equations. For example, using known values of RGB, Equations 1-3 can be solved to derive YUV values.

Y = 0.30R + 0.59G + 0.11B (Formula 1)
U = 0.70R-0.59G-0.11B (Formula 2)
V = −0.3R−0.59G + 0.89B (Formula 3)

  The U and V values do not include a luminance component. If the values of R, G, B are exactly the same, a gray color is defined and the corresponding U and V values are zero. Further, the color does not change corresponding to the change of the Y value, and the luminance does not change corresponding to the color change. In other color space systems, Y is the luminance, and Yxy coordinates where x and y represent colors on the tilted parabola color space chromaticity diagram are used. In this Yxy coordinate, the outer curved portion is a spectral (or monochromatic) locus. It is also possible to modify the techniques described herein to use a Yxy display that shows a set of pixel values instead of a YUV display. The YUV color display can be determined in response to a special sensor 54 included in the camera assembly 12, and the conversion between brightness and color is adjusted to provide a dedicated camera assembly 12 correction. Note that it is possible.

  At block 68, a determination is made as to whether the pixel corresponds to a skin color portion of the image. For example, when the maximum value of Y is 255, typical skin colors have YUV values of 188, 42, -32 or 152, 22, -27. In one embodiment, a pixel can be determined to be associated with a skin color if the pixel's UV value is located in a predetermined rectangular portion of the UV plane. For example, the skin color can be defined as having a U value between 21 and 43 and a V value between −22 and −33. In other embodiments, the portion of the UV plane associated with the skin color may be non-rectangular, such as an ellipse or a set of continuous squares that approximate an ellipse. For example, skin color can be associated with one or more minimum discrimination values (JND) centered at certain UV coordinates. For example, when the Yxy color value is used, the skin color can be determined using five JND (5 × JND) ellipse sizes centered at xy coordinates of 0.37 and 0.27. Values contained in one or more JND regions can be obtained from publicly available sources. The range of values associated with skin color for the purpose of driving camera photometry is for a particular camera assembly (eg to account for sensor configuration variations) and / or a consumer market for a particular camera. Adjustments can be made.

  Those skilled in the art will appreciate that based on the above technique, it is possible to identify skin color pixels of almost any person, regardless of race. That is, European offspring, African offspring and Asian offspring generally have the same skin color, but differ in the amount of skin pigment. UV and xy color value schemes can be used to detect skin tone affected by pigment levels. However, it should be noted that an overexposed or underexposed image of the skin may not be identified as containing skin color. For example, overexposed skin of a white race subject may have YUV values 253, -2, -2 where a maximum Y value of 255 exists. Accordingly, a positive determination may not be obtained at block 68 for a portion of the skin that is lightly reflective off and extremely shaded. As will become more apparent below, excluding such pixels from the skin color region will result in one or more identifiable skin color spots and corresponding improvements when establishing exposure settings related to lighting conditions. It may lead to driving of spot metering performed based on this.

  If a negative determination is made at block 68, processing proceeds to block 70. In block 70, the pixel is tagged as not containing skin color. Apart from the above, it is also possible to “zero” the pixel, for example by setting the luminance value and color value of the pixel to be all represented in black. If a positive determination is made at block 68, the process proceeds to block 72 where the pixel is tagged as containing skin color. It should be appreciated that a pixel can be associated with a person's skin or similarly colored object. Subsequent processing is used to identify potential body portion structures that may be useful in driving photometry of the field of view of the camera assembly 12.

  Following blocks 70 and 72, processing proceeds to block 74 where a determination is made as to whether the last pixel obtained from the output side of sensor 54 has been subjected to the skin color pixel identification process of block 62. In one embodiment, all pixels obtained from the field of view of the camera assembly 12 are subject to a skin color pixel identification process. In other embodiments, the selected set of pixels is subject to a skin color pixel identification process. For example, it is assumed that most users tend to place a human subject at the center of a photograph. Accordingly, the portion of the field outside the central region of the field of view of the camera assembly 12 can be excluded during the flesh color pixel identification process and during the subsequent flesh color pixel analysis described below. If a negative determination is made at block 74, processing returns to block 64. If an affirmative determination is made at block 74, processing proceeds to the skin color pixel analysis of the method.

  With continued reference to FIGS. 5 and 6, once the flesh color pixel is identified in block 62, the process proceeds to block 76 where the flesh color pixel is analyzed. A detailed approach to skin color pixel analysis is shown in FIG. This analysis determines whether it is desirable to drive the photometry of the field of view of the camera assembly 12 using one or more skin tone groups of tonal pixels to lead to the determination of exposure settings. .

  The process for analyzing skin color pixels starts at block 78, in which the pixels identified in block 72 are grouped as containing skin color. This grouping can be determined by associating individual skin color pixels with other skin color pixels that are separated by a distance less than a predetermined distance. Groups over a predetermined distance can be established due to multiple associations with pixels. The area occupied by this group need not be contiguous (for example, the area of the group may include pixels that do not have skin color).

  Individual groups can have associated shapes. Although the body part of the subject may not have a clearly defined shape, it will still occupy the field of view of the camera assembly 12, which can be roughly identified as a normal shape. For example, the face has an oval shape, the limbs (such as arms or legs) not covered by clothing have an elongated shape, and the torso not covered by clothing has a square shape The body part exposed by the V-shaped neckline has a triangular shape, etc. In addition, a pixel corresponding to an inanimate object including skin color may have a geometric shape corresponding to the object.

  In block 80, the group of skin color pixels is evaluated against each other or an identification criterion to identify one of a plurality of skin color spots used to drive camera photometry. Skin color pixels grouped by isolated skin color pixels or other skin color pixels less than a certain threshold number (eg, about 20 threshold values) are suitable for human subjects suitable for driving camera photometry. It may be ignored as it relates to a region that is too narrow to represent a part. In one embodiment, a group of flesh-colored pixels having a well-defined geometric shape (eg, as defined by a straight line or uniform curve as measured by a perimeter) is in an area corresponding to an inanimate subject. Ignored as such.

  In the analysis, the skin color pixel groups are made to include different tones so that they can be adapted to human eyes, lips, teeth, facial hair, makeup, jewelry and the like. In one embodiment, if the area containing the flesh tone pixel group has pixels that are greater than or equal to a predetermined threshold percentage, a portion of one or more people is placed in a corresponding spot in the field of view of the camera assembly 12. You may conclude that In one embodiment, the threshold may be 50%. In other embodiments, the threshold may be 60%. In other embodiments, the threshold may be 70%. And in one other embodiment, the threshold may be 80%. A pixel group whose skin color pixels are less than the threshold ratio may be ignored for the convenience of camera photometry, or may be given a lower weight than the skin color spot.

  The remaining plurality of groups of pixels can be analyzed to select one or more groups for use in camera photometry. For example, the basic setting can be performed for the skin color region arranged at the position closest to the center of the visual field of the camera assembly 12. For the convenience of photometry, a higher weight is given to the skin color group arranged at the position closest to the center of the field of view than the other skin color regions. Alternatively, more end groups may be ignored during photometry. As another example, a basic setting for a face can be created in preference to limbs and torso. In the present embodiment, it is assumed that one or more groups positioned above one or more other groups can be selected for use during camera photometry in a state in which the person to be photographed is faced up. As another method for setting the basic setting for the face in preference to the limbs and the torso, there is generally a method for selecting the oval group in preference to the other shape group. In one approach, photometry may be performed under these methods, ignoring groups not related to the face. Alternatively, in other approaches, groups that are not related to faces under these methods may be included during photometry, but will be given a lower weight than groups related to faces.

  As the exemplary method shows, the specific process of block 80 can reduce the number of skin color groups to use during camera photometry, or determine the weighting rank to give to each group. In other embodiments, all pixels or all pixels associated with a threshold number of pixel groups may be used for photometry regardless of relative size, shape, position, proportion, etc. Also good. In any of these approaches, if more than one skin tone spot is identified, select one of those skin tone spots and select some of the skin tone spots to use for camera photometry Then, it can be used for camera photometry, or all skin color spots can be selected and used for camera photometry. As shown in the figure, when a plurality of skin color spots are used for camera photometry, different weights for photometry can be given to the skin color spots depending on at least one of the relative size, shape and position of the spots.

  Returning to the flowchart of FIG. Processing continues at block 82. In block 82, a determination is made as to whether any flesh spot identified through the analysis of block 76 is suitable for use with camera photometry. If a negative determination is made, the process proceeds to block 84, where camera photometry is performed by a conventional method such as conventional spot metering or area metering.

  If an affirmative determination is made at block 82, the process proceeds to block 86 where the size of one or more skin color spots used for camera photometry is changed or shaped. By the step of changing the size or shaping, it becomes possible to configure an array of one or more spots and easily perform processing associated with photometry. For example, squares or other polygons can be handled more easily than irregular shapes associated with flesh-colored spots. Also, one or more spot regions can be narrowed (eg, by about 5% to about 15%) to remove pixels at the peripheral edge of the one or more spots. In this way, one or more spot edges are ignored in order to perform photometry. This makes it possible to compensate for the fact that the color resolution of many cameras is not as good as luminance detection. Therefore, a slow color transition to the skin color occurs at the edge of the face, and the removal of the edge of the skin color spot results in the spot more accurately over the skin area of the subject having the desired illumination quality when driving camera metering. Will be wrapped.

  Next, in block 88, camera photometry is performed based on the identified skin color spot. Here, for example, spot photometry can be employed. During spot metering, it is possible to monitor the illumination level across the entire field of view (or part of the field) of the camera assembly 12. The lighting levels are integrated together to evaluate the lighting conditions. During the integration of the illumination level, the specified one or more skin color spots are given higher weight than the non-skin color region. Such spot photometry may be combined with area photometry. As will be appreciated, illumination associated with one or more skin tone spots will affect the photometric result. Therefore, as a photometric result, more accurately than a case where a predetermined spot is used with respect to a normal spot and an area photometry method is used, such as a person in the camera's field of view It is possible to represent the lighting conditions associated with a part of the picture of the user's primary interest.

  Following blocks 84 and 88 (depending on the result of block 82), processing proceeds to block 90. In this block 90, the exposure setting value is adjusted using the photometric result. Exemplary exposure settings that can be adjusted according to photometry may include values that indicate whether or not the flash 60 is used, and (if adjustable) flash intensity, aperture value, shutter speed, etc. May be included.

  In one embodiment, the exposure setpoint can be adjusted to meet certain criteria. By this adjustment, the extreme value of luminance becomes a value that is neither too dark nor too bright. For example, it is desirable that skin color pixels having a predetermined ratio or less have a luminance value equal to or higher than a predetermined upper luminance threshold value, and skin color pixels having a predetermined ratio or lower have a luminance value equal to or lower than a predetermined lower luminance threshold value. It is desirable to have In other words, after adjusting the exposure setting value between the upper and lower luminance threshold values, it is desirable that a certain proportion of skin color pixels contain a certain luminance value. In one example, for a 24-bit image in which the maximum value of Y is 255 and the minimum value is 0, it is desirable that 5% or less flesh-colored pixels have a luminance value higher than 225, and 25% The following skin color pixels desirably have a luminance value of less than 80. Depending on the environment, if one or two of the above criteria are also exceeded, there may be a probability that there is a significant number of pixels associated with human skin that cannot be identified in block 68. This is because these pixels are either in a supersaturated state (the reflectance is too high to loosen the skin color) or in an unsaturated state (the reflectance is too low to loosen the skin color) Because it is.

  Furthermore, a limit may be imposed on the amount of change in luminance caused by adjustment to the exposure setting value. By this limit, it is possible to minimize or adjust the adjustment value from the average value of the image that exceeds a certain amount by saturation or dimming of the skin color spot. This is because if the exposure is too bright or too dark, the color information used to identify the skin color will be lost.

  In one embodiment, iterative processing can be employed as part of adjusting exposure settings. For example, an optimum exposure setting value may be found using a step of incrementing the exposure setting value until skin color saturation is detected. In one approach, the process begins with an exposure setting that produces low image brightness when the user depresses or releases a shutter button that commands photography. Next, an adjustment is made to gradually increase the exposure setting value in order to increase the luminance. Corresponding to each increment (or step) value, the flesh color spot can be specified as described above, and then photometry can be performed. When the predetermined proportion of skin color pixels detected in advance shifts toward saturation, the increment step is terminated. In one implementation, after the increment step is finished, the exposure setting is the resulting exposure setting when skin tone saturation is detected (e.g., this exposure setting is reversed by one or more increments). The image may be slightly darker than the one performed. This incremental approach increases the number of flesh-color pixels in the image used for photometry and reduces the likelihood that flesh-color pixels will not be identified due to glare. Preferably, it is desirable to be able to end the repetitive adjustment process of exposure settings before the user moves the camera assembly 12. This results in a change in the field of view of the camera assembly 12 or a significant amount of movement of the human subject's field of view.

  After the exposure setting value is adjusted, a photograph is taken in block 92.

  In the method described in this specification, spot photometric weighting is performed by skin color identification. As a result, the exposure setting is performed, and the skin color can be optimally arranged in the dynamic range of the camera assembly 12. In this way, the faces and other areas associated with the person appearing in the field of view of the camera assembly 12 are higher in the resulting photograph than other subjects in the field of view, such as backgrounds and foreground objects. You will have detailed level information. As a result, the user can take a photograph including a person who has been well imaged without manually adjusting the exposure setting value or performing post-processing of the photograph.

  In the method described in this specification, a search is performed by obtaining a pixel including a color related to the skin color and by obtaining a skin color pixel group indicating a part of a human subject. Using these groups of pixels, it is possible to execute a photometric and exposure adjustment value adjustment process. As will be appreciated, the identification of one or more flesh-colored spots that are given higher weight than other parts of the field of view of the camera assembly can be accomplished by searching part or all of the color space of the field of view to provide sufficient area and field of view. Thus, it is possible to find a pixel that combines at least one of the above in the storage position within a certain color range and display a highly probable human subject. As a result, it is possible to identify a suitable spot and guide the adjustment process of spot metering and exposure setting.

  While the invention has been illustrated and described in connection with preferred embodiments, equivalents and modifications will occur to those skilled in the art upon reading and understanding this specification. The present invention includes all such equivalents as well as modifications. The invention is specified by the claims.

Claims (10)

A method for adjusting an exposure setting value of a digital camera (12),
Searching for flesh-colored pixels from at least part of the camera's field of view;
Analyzing the flesh color pixels to identify at least one flesh color spot;
Performing spot metering of the field of view of the camera by providing at least one skin tone spot with a higher weight than the rest of the field of view;
Adjusting the exposure setting value based on the spot metering;
A method characterized by comprising:   The method of claim 1, wherein the analysis of the flesh color pixels includes grouping the flesh color pixels by relative approximation between pixels.   3. The method according to claim 1, wherein the skin color spot region has skin color pixels at least in a threshold ratio.   4. The method according to claim 1, further comprising the step of reducing the size of the flesh color spot before performing the spot photometry.   5. The method according to claim 1, further comprising the step of converting the RGB output values (54) of the sensor that images the field of view into coordinate-based color values of individual pixels. .   6. A method according to any one of the preceding claims, wherein the skin color is defined by a predetermined portion of a coordinate-based color space.   7. The method according to claim 1, further comprising the step of repeatedly adjusting the exposure setting value from dark to bright until skin color saturation is detected.   The method according to claim 1, further comprising executing at least a part of the iterative adjustment in a reverse direction when skin color saturation is detected. A camera (12),
A sensor (54) for imaging the field of view and outputting signals indicating luminance and color for a plurality of pixels;
Identifying a flesh color pixel; analyzing the flesh color pixel to identify at least one flesh color spot; assigning at least one flesh color spot a higher weight than the rest of the field of view; A controller (52) for adjusting an exposure setting value of the camera by executing a step of adjusting the exposure setting value based on the spot metering;
A camera (12) characterized by comprising:   Camera according to claim 9, characterized in that it forms part of a mobile radio terminal (10) comprising a radio circuit (26) for establishing a call via a communication network (48).

Images