JP2009297233A - Pulse wave detector, pulse wave detection program and pulse wave detection method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a pulse wave detector highly accurately measuring pulse waves, a pulse wave detection program and a pulse wave detection method. <P>SOLUTION: The pulse wave detector calculates a luminance average, an edge amount in a horizontal direction or a vertical direction, the evaluation value of a luminance distribution, and the evaluation value of the centroid of an input image on the basis of the input image captured by placing a finger on a lens of a camera, calculates evaluation points by comparing each of respective calculated characteristics with a prescribed threshold, determines whether or not the way of placing a finger is correct on the basis of the respective calculated evaluation points, outputs an error when it is determined that the way of placing the finger is not correct, and calculates and outputs a pulse on the basis of signals obtained from the input image when it is determined that the way of placing the finger is good. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a pulse wave detection device, a pulse wave detection program, and a pulse wave detection method.

  In recent years, an increasing number of people are interested in equipment for health care, and not only body fat meters and pulse wave meters, but also various health care that can be owned by individuals such as electrocardiographs that can easily measure electrocardiograms. The equipment is increasing. For example, as a pulse wave meter, there is a clip-type sensor that is sandwiched between a finger or an earlobe.

  Regarding the above-described pulse wave meter, it is necessary to purchase one of the health management devices such as the pulse wave meter itself, an exercise bike with a pulse wave sensor, and a pedometer (registered trademark). Therefore, recently, a technique for measuring a pulse wave with a camera mounted on a mobile terminal such as a mobile phone or a PDA (Personal Digital Assistants) is desired.

  A typical pulse wave sensor is an active sensor that combines a light-emitting element and a light-receiving element, and has a mechanism that blocks external light by wrapping a belt around a finger or pinching a finger or ear with a clip. Less susceptible to light. Similarly, when measuring a pulse wave with a camera mounted on the portable terminal, it is necessary to place a finger at an appropriate position in order to reduce the influence of external light.

  As a technique for placing the user's finger at the appropriate position of the camera, the finger is brought into contact with the cover of the CCD (Charge Coupled Devices) camera, and the pressing surface is moved and pressed (the pressed state). A technique for collecting a fingerprint is disclosed (see Patent Document 1). That is, the disclosed technology appropriately guides the position where the user's finger is placed by using a structure in which the pressing surface is movable.

JP-A-11-225998

  However, the above-described prior art has a problem in that pulse waves cannot be measured with high accuracy. Specifically, the above-described prior art is intended for fingerprint collection, and since the time for collection is instantaneous, it is highly accurate when used for pulse wave measurement that requires multiple cycles. Cannot be measured.

  Accordingly, the present invention has been made to solve the above-described problems of the prior art, and a pulse wave detection device, a pulse wave detection program, and a pulse wave detection method capable of measuring a pulse wave with high accuracy. The purpose is to provide.

  In order to solve the above-described problems and achieve the object, a pulse wave detection device disclosed in the present application is a pulse wave detection device that measures a pulse wave based on an image captured by an imaging unit, When it is a mode used for acquisition, it is determined whether the state of the image is one of luminance, an edge amount, a center of gravity of the sum of luminance values in the vertical direction and the horizontal direction of the image frame, or a plurality of the predetermined states. When the determination result of the determination unit and the determination unit are not in a predetermined state, an error is output, and when it is in the predetermined state, the pulse is calculated based on the signal obtained from the image, and the calculation result is And an output unit for outputting.

  According to the pulse wave detection device, the pulse wave detection program, and the pulse wave detection method disclosed in the present application, it is possible to measure the pulse wave with high accuracy.

  Exemplary embodiments of a pulse wave detection device according to the present invention will be described below in detail with reference to the accompanying drawings.

[Overview of pulse wave detector]
Initially, the outline | summary of the pulse-wave detection apparatus concerning Example 1 is demonstrated. The pulse wave detection device disclosed in the present application is disposed in a mobile terminal such as a mobile phone with a camera or a PDA, for example. Normally, a camera disposed in such a portable terminal is, for example, a general camera mode for capturing a landscape or the like, and a pulse wave acquisition as disclosed in the present embodiment. It has at least two types of modes, a pulse wave acquisition mode using a camera in the application. Of course, when a camera is used exclusively for pulse acquisition, it may have only one type of pulse wave acquisition mode.

  The pulse wave acquisition mode may not be preset in the camera. For example, it is suitable for pulse wave acquisition by changing camera settings (exposure time, frame rate, etc.) from the pulse wave detection program. The set state may be regarded as the pulse wave acquisition mode. For example, even if the frame rate is fixed, it is not necessary to clearly distinguish between the normal imaging mode and the pulse wave acquisition mode when the frame rate can be used for both the normal imaging application and the pulse wave acquisition application.

  The pulse wave detection device measures a pulse wave based on an image photographed by placing the user's finger on the camera lens, and in particular, appropriately positions the user's finger placed on the camera lens. It is possible to measure the pulse wave with high accuracy by instructing to.

  In the configuration described above, when the pulse wave detection device is in a mode in which an image captured with a finger placed on the lens of the camera is used for pulse acquisition, the state of the image is represented by luminance, edge amount, and vertical direction of the image frame. It is determined whether one or more of the centroids of the luminance value sum in the horizontal direction are in a predetermined state. The pulse wave detection device outputs an error message when the determination result is not in a predetermined state, calculates the pulse based on the signal obtained from the image when the determination result is in the predetermined state, and outputs the calculation result To do.

  Specifically, the pulse wave detection device accepts an input image that is captured with the user's finger placed on the lens of the camera. Then, the pulse wave detection device calculates a luminance average from the luminance sum total of the input image. Subsequently, the pulse wave detection device calculates an edge amount in the horizontal direction when the flicker direction of the camera is vertical, and in the vertical direction when the flicker is horizontal.

  Further, the pulse wave detection device calculates the evaluation value of the luminance distribution from the average luminance of the input image. Further, the pulse wave detection device calculates the center of gravity position in the vertical direction and the horizontal direction from the total luminance value in the vertical direction and the total luminance value in the horizontal direction of the input image, and evaluates based on the calculated center of gravity. Is calculated. Note that the above-described characteristics (luminance average, edge amount, luminance distribution, and center of gravity) are calculated for each input image (for each frame).

  Thereafter, the pulse wave detection device calculates an evaluation score by comparing each of the calculated luminance average, edge amount, luminance distribution, and center of gravity with a predetermined threshold value. Then, the pulse wave detection device determines whether the user's finger is placed in a good state based on the calculated evaluation points, and further determines whether the ambient light is in an appropriate state based on the luminance average To do.

  Subsequently, the pulse wave detection device outputs an error message when the user's finger placement is poor, or when the external light is not appropriate, and the user finger placement is good. The pulse is calculated and output based on the signal obtained from the input image. The determination of the user's finger placement state uses the accumulation of each characteristic calculated for each frame of the input image, that is, each characteristic for a plurality of frames.

  Further, when the frame rate of the acquired image is sufficiently high, the frames to be processed may be thinned out, for example, every other frame without calculating each characteristic for each frame. Further, the accumulation of each characteristic may be the accumulation from the start of measurement, or the characteristic from the timing of calculating the accumulation to the specified time.

  As described above, when the pulse wave detection device measures a pulse wave based on an image photographed by the camera, is the user's finger placed on the camera lens based on the characteristics of the input image? It is possible to measure the pulse wave with high accuracy as a result of determining whether or not and outputting an error when the user's finger placement is poor.

  A pulse wave detection device disposed in a mobile terminal or the like is a passive sensor that receives external light transmitted through a user's finger with a camera. If the finger position is poor due to slight movement of the finger, the external environment is reflected. In other words, the light transmission amount fluctuates greatly, and the pulse wave information is buried in noise. That is, in order to accurately measure the pulse wave, the position where the finger is placed is important. In other words, the pulse wave detection device determines whether or not the finger is placed properly by evaluating each characteristic of the image taken when the finger is placed. As a result of outputting an error, it is possible to measure a pulse wave with high accuracy.

[Configuration of pulse wave detector]
Next, the configuration of the pulse wave detection device according to the first embodiment will be described with reference to FIG. FIG. 1 is a diagram illustrating a configuration example of a pulse wave detection device according to the first embodiment.

  As shown in FIG. 1, the pulse wave detection device 10 includes an output unit 20, a storage unit 30, and a control unit 40. For example, the pulse wave detection device 10 is disposed in a mobile terminal such as a mobile phone with a camera or a PDA, A pulse wave is measured based on an image taken by placing a user's finger on the lens of the camera.

  The output unit 20 includes a monitor (or a display, a touch panel, etc.) and a speaker, and outputs various types of information. In particular, the error display unit 21, the pulse rate display unit 22, the file 23, and the pulsation display unit 24 are output. And have. For example, the output unit 20 displays and notifies various processing results by the control unit 40 to the user, and outputs data obtained from the various processings by the control unit 40 as a file.

  When an error occurs as a result of various processes by the control unit 40, the error display unit 21 displays the error content. For example, if the error display unit 21 determines that the position of the user's finger placed on the camera of the mobile terminal is bad by an error determination unit 42a described later, “the finger is not placed”, “how to place the finger” "Error is bad" is displayed.

  The pulse rate display unit 22 displays the pulse rate calculated by the pulse rate calculation unit 42c described later. The file 23 outputs the pulse rate data calculated by the pulse rate calculation unit 42c described later as a file. The pulsation display unit 24 displays pulsations detected by a peak detection unit 42b described later.

  The storage unit 30 stores data necessary for various types of processing by the control unit 40 and various types of processing results by the control unit 40, and in particular, a luminance information storage unit 31, an error information storage unit 32, and a peak information storage unit 33. And have. For example, the storage unit 30 stores an image processing result for each image processed by the image processing unit 41 described later, and stores waveform data, error information, and the like processed by a waveform processing unit 42 described later. To do.

  The luminance information storage unit 31 stores the luminance average of the input image calculated by the luminance average calculation unit 41b described later. Further, the error information storage unit 32 stores an evaluation score for each input image calculated by an evaluation score calculation unit 41f described later. The peak information storage unit 33 stores pulse wave information based on an input image detected by a peak detection unit 42b described later.

  The control unit 40 has an internal memory for storing a control program, a program that defines various processing procedures, and necessary data, and in particular, an image processing unit 41 and a waveform processing unit 42, thereby Various processes are executed.

  The image processing unit 41 performs image processing for each input image, and in particular, a preprocessing unit 41a, a luminance average calculation unit 41b, an edge amount calculation unit 41c, a luminance distribution calculation unit 41d, and a centroid It has the calculation part 41e and the evaluation score calculation part 41f.

  The pre-processing unit 41a receives an image captured by placing a user's finger on the camera lens of the mobile terminal, extracts a luminance component from the received image, or reduces the received image.

  More specifically, the pre-processing unit 41a receives image data “G0” captured by placing a user's finger on the lens of the camera of the mobile terminal, and extracts a luminance component from the received image data “G0”. Extract. Further, the preprocessing unit 41a reduces the received image data “G0” to generate image data “G1”. The data to be used is not limited to the luminance component as long as the pulse wave can be captured. For example, the R component of RGB (Red-Green-Blue) or YUV (luminance signal: Y, luminance signal) And the blue component: U, and the difference between the luminance signal and the red component: V) may be the V component.

  Here, the feature of the image corresponding to how to place a finger on the lens of the camera of the mobile terminal will be described with reference to FIG. FIG. 2 is a diagram for explaining how to place a finger and image characteristics. In FIG. 2, (1) dark, (2) normal, (3) bright, (4) poor finger placement, (5) very poor finger placement, (6) finger placement. (A) Original image, (b) Luminance information, (c) Vertical edge, (d) Horizontal edge, (e) 8 × 8 area average (the image is a block unit of 8 × 8 pixels) And an average of each area after the division).

(1) Darkening An image (original image) captured by placing a user's finger on the lens of the camera of the mobile terminal is a dark image as shown in (a) of (1) of FIG. In this case, the characteristics of (b) to (e) are generally dark, the luminance component is uniform, and there are almost no edges. If the image is dark, the position where the finger is placed on the lens of the camera of the mobile terminal is appropriate, but the ambient light is weak, such as when the illumination is dark.

(2) Ordinary An image taken by placing the user's finger on the camera lens of the mobile terminal is an image having a normal (appropriate) brightness as shown in (a) of (2) of FIG. In this case, in (b) to (e), in particular, it is dark as a whole, the luminance component is almost uniform, and there are almost no edges. When the image is normal, the position where the finger is placed on the lens of the camera of the mobile terminal and the ambient light are appropriate.

(3) Bright When the image taken by placing the user's finger on the lens of the camera of the mobile terminal is a bright image as shown in (a) of (3) of FIG. In (e), the influence of flicker (“flicker” in the image) may be particularly noticeable at the edges in the horizontal direction. When the image is bright, the position where the finger is placed on the lens of the camera of the mobile terminal is appropriate, but the ambient light is strong, such as directly under illumination or under hot weather.

(4) Poor finger placement When the image taken by placing the user's finger on the camera lens of the mobile terminal is an image as shown in (a) of (4) of FIG. In (b) to (e), a part of the 8 × 8 region is particularly brightened. Note that when the image is (4) in FIG. 2, the position where the finger is placed on the lens of the camera of the mobile terminal is not appropriate, and external light (external light) is reflected. A part becomes bright.

(5) When the finger is placed very badly When the image taken by placing the user's finger on the lens of the camera of the mobile terminal is an image as shown in (a) of (5) of FIG. In (b) to (e), in particular, a part of the 8 × 8 region becomes brighter and the edges increase. When the image is (5) in FIG. 2, the position where the finger is placed on the lens of the camera of the mobile terminal is not appropriate, so external light is reflected (compared to (4) in FIG. 2). In addition, since external light is reflected), as described above, a part becomes brighter and edges also increase.

(6) When a finger is not placed When the photographed image is an image as shown in (a) of (6) of FIG. 2, in (b) to (e), the entire image is particularly bright. , The edge will be very much. When the image is (6) in FIG. 2, since no finger is placed on the lens of the camera, external light is reflected in the entire area, so that the image is bright as a whole as described above and the edge Also very much.

  And the point which determines the state of how to put a finger as shown in FIG. 3 can be derived | led-out from the image pattern in the various situations shown in FIG. FIG. 3 is a diagram for explaining points for determining a state of placing a finger.

  For example, when the position of the finger placed on the lens of the camera is normal (appropriate), as shown in FIG. 3, the brightness “dark”, the distribution (luminance distribution) “uniform”, and the edge “flicker” There is a tendency to “nearly” and “center of gravity”.

  Also, for example, when the position of the finger placed on the lens of the camera is bad, as shown in FIG. become.

  Further, for example, when no finger is placed on the lens of the camera (when the outside world is photographed), as shown in FIG. 3, the brightness is “bright”, the distribution is “entirely bright”, and the edge is “many”. The center of gravity tends to be “no trend”.

  In addition, when the finger is placed in a bad manner (see FIG. 2 or FIG. 3), the waveform becomes bright because external light is reflected as in the region where the finger on the left side of the image is not placed as shown in FIG. Since there are a portion and a portion that becomes dark because the finger is placed, the waveform has a difference in luminance average over time. FIG. 4 is a diagram illustrating an example of an image and a waveform when a finger placed on the lens of the camera is poorly placed.

  Furthermore, the intensity of external light and the tendency in the image will be described in detail with reference to FIGS. In these figures, the vertical axis represents the luminance value and the horizontal axis represents time. First, the illuminance and luminance average around the camera will be described with reference to FIGS. 5A is a diagram illustrating an example of illuminance around the camera, and FIG. 5B is a diagram illustrating an average luminance example of an image corresponding to the illuminance around the camera.

  For example, the waveform of the portion with a high average brightness (bright) shown in FIG. 5A is a beautiful waveform as shown in FIG. Further, for example, a waveform of a portion where the luminance average shown in FIG. 5-2 (B) is very low (dark) and a portion where the luminance average shown in FIG. 5-2 (C) is low (dark). As shown in FIGS. 5-4 and 5-5, respectively, the waveform becomes dirty compared to FIG. 5-3. 5-3 is a diagram illustrating an example of a waveform when the image is bright, FIG. 5-4 is a diagram illustrating an example of a waveform when the image is dark, and FIG. 5-5 is a diagram where the image is dark. It is a figure which shows the example of a waveform in a case.

  In short, the pulse wave detection device 10 disposed in a portable terminal or the like is a passive sensor that receives external light transmitted through a finger with a camera, and noise increases when the amount of light transmission is small. Number calculation becomes difficult.

  Returning to FIG. 1, the luminance average calculation unit 41b calculates the luminance average of the image data generated by the preprocessing unit 41a. Specifically, the luminance average calculation unit 41b calculates the luminance sum “ttlY” of the image data “G1” generated by the preprocessing unit 41a. Then, the luminance average calculation unit 41 b calculates the luminance average “YMean” of the image data “G1” from the calculated luminance sum “ttlY” of the image data “G1”, and stores it in the luminance information storage unit 31.

  Note that the luminance average “YMean” calculated by the luminance average calculation unit 41b may be a luminance sum “ttlY” or a constant multiple of the luminance sum “ttlY”. When the luminance sum “ttlY” is used, , The processing load can be reduced.

  The edge amount calculation unit 41c calculates the sum of the edge amounts in the vertical direction or the horizontal direction of the image data generated by the preprocessing unit 41a. Specifically, the edge amount calculation unit 41c in the image data “G1” generated by the preprocessing unit 41a has a horizontal edge “Edge_X” and flickers appear horizontally when flicker appears vertically. In this case, the vertical edge “Edge_Y” is calculated. Then, the edge amount calculation unit 41c calculates the total edge amount “SEEdge” of the calculated horizontal edge “Edge_X” or vertical edge “Edge_Y”.

  Since the flicker is determined in advance by the camera module, the edge amount calculation unit 41c determines which one of the required edge amounts of the horizontal edge “Edge_X” or the vertical edge “Edge_Y” based on the flicker. The sum “SEdge” is calculated. Note that flicker may appear when taken directly under a fluorescent lamp or the like (for example, see FIG. 2 (3)).

  The luminance distribution calculation unit 41d divides the image data generated by the preprocessing unit 41a into predetermined regions, calculates an average luminance of each region, and calculates an evaluation value of the luminance distribution. More specifically, the luminance distribution calculation unit 41d divides the image data “G1” generated by the preprocessing unit 41a into M × N regions, and calculates the luminance average Y (m, n) of each region. ) Is calculated.

  Then, the luminance distribution calculation unit 41d calculates the luminance average “, which is an average of all the calculated luminance averages Y (m, n), the maximum value“ YMax ”, the minimum value“ YMin ”, and all Y (m, n). YMean ”is calculated, and the evaluation value“ Dst = (YMax−YMin) × M × N ÷ YMean ”of the luminance distribution is calculated. The luminance average YMean used in the luminance distribution calculation unit 41d may use the luminance average calculation result calculated by the luminance average calculation unit 41b.

Also, the evaluation value “Dst” is calculated using an appropriate constant “K”.
Dst = (YMax−YMin) × K
Dst = (YMax−YMin) × K ÷ YMean
Dst = YMax × K
Dst = YMax × K ÷ YMean
It may be calculated as follows.

  The centroid calculation unit 41e calculates the centroid position for each of the horizontal direction and the vertical direction of the image data generated by the preprocessing unit 41a. Then, the centroid calculating unit 41e calculates an evaluation value from the calculated centroid positions in the horizontal direction and the vertical direction.

Specifically, in the above-described example, the center-of-gravity calculation unit 41e performs the horizontal direction and the vertical direction of the image data “G1” generated by the preprocessing unit 41a as shown in FIG. And the center-of-gravity positions “Cx” (horizontal direction) and “Cy” (vertical direction) are calculated from (Expression 2). In (Expression 1) and (Expression 2), “G _N ” is “number of pixels = (W1 + W2 + 1) × (H1 + H2 + 1)”, and “G (i, j)” is luminance (0 to 1). . FIG. 6 is a diagram for explaining how to obtain the center of gravity of an image.

  Subsequently, the center-of-gravity calculation unit 41e calculates the evaluation value of the center of gravity of the image data “G1” using (Expression 3) from the calculated horizontal center-of-gravity position “Cx” and vertical center-of-gravity position “Cy”. Calculate “cPos”. Note that the origin of the horizontal gravity center position “Cx” and the vertical gravity center position “Cy” is the center (0, 0) of the image shown in FIG.

  Note that the horizontal center of gravity position “Cx” and the vertical center of gravity position “Cy” of the image data “G1” are the sum of the horizontal brightness “Sx (i)” (Formula 4) and the vertical brightness. May be calculated from the sum “Sy (i)” (Equation 5) (Equation 6) and (Equation 7).

  The evaluation point calculation unit 41f compares the values calculated by the luminance average calculation unit 41b, the edge amount calculation unit 41c, the luminance distribution calculation unit 41d, and the centroid calculation unit 41e with a separately determined threshold value to obtain an evaluation score. Calculate and store in the error information storage unit 32.

  More specifically, the evaluation point calculation unit 41f compares the total edge amount “SEEdge” calculated by the edge amount calculation unit 41c with the threshold “Th_Edge1” (edge amount threshold). When the sum of the edge amounts is larger than the threshold value, the evaluation score is “P1_1”.

  Then, the evaluation point calculation unit 41f, when the total edge amount “SEEdge” is smaller than the threshold “Th_Edge1”, the luminance average “YMean” calculated by the luminance average calculation unit 41d and the threshold “Th_yD1” (is dark (Threshold threshold). Subsequently, the evaluation score calculation unit 41f sets the evaluation score “P1_2” when the average brightness is smaller than the threshold value.

  Thereafter, when the luminance average “YMean” is larger than the threshold value “Th_yD1”, the evaluation score calculation unit 41f and the luminance average “YMean” calculated by the luminance average calculation unit 41b and the threshold value “Th_yB1” (whether the luminance is bright) (Threshold). Then, the evaluation score calculation unit 41f sets the evaluation score “P1_3” when the luminance average is smaller than the threshold value.

  Subsequently, when the average luminance “YMean” is larger than the threshold value “Th_yB1”, the evaluation point calculation unit 41f calculates the luminance distribution evaluation value “Dst” calculated by the luminance distribution calculation unit 41d and the threshold value “Th_Dst1” ( (Luminance distribution threshold). Thereafter, the evaluation score calculation unit 41f sets the evaluation score “P1_4” when the evaluation value of the luminance distribution is larger than the threshold value.

  Then, when the evaluation value “Dst” of the luminance distribution is smaller than the threshold value “Th_Dst1”, the evaluation point calculation unit 41f and the evaluation value “cPos” of the centroid calculated by the centroid calculation unit 41e and the threshold value “Th_cPos1” ( The threshold of the center of gravity). Subsequently, the evaluation score calculation unit 41f sets the evaluation score “P1_5” when the evaluation value of the center of gravity is larger than the threshold value. The evaluation score calculation unit 41f sets the evaluation score “P_OK” when the evaluation value “cPos” of the center of gravity is smaller than the threshold “Th_cPos1”.

  Note that the evaluation score by the evaluation score calculation unit 41f is not determined in one stage as described above, but can be further divided into several stages to determine the evaluation score. Evaluation points can be determined.

  Further, each threshold value used in the evaluation point calculation unit 41f is preferably obtained in accordance with the characteristics of the camera used for pulse wave acquisition. As a method for setting each threshold, a finger is placed on a camera used for pulse wave acquisition (a camera having the same characteristics) as in (1) to (3) in FIG. Find the brightness that makes it difficult to pick. Then, an image group (referred to as an image group “A”) when a finger is appropriately placed as in (1) to (3) in FIG. 2 and (4) to (6) in FIG. A plurality of data is acquired by dividing into an image group (referred to as an image group “B”) when the finger is not properly placed.

  Subsequently, the edge amount, the luminance distribution, and the data of the center of gravity are compared between the image group “A” and the image group “B”, and a threshold value is set in the middle so that the point (evaluation point) becomes higher as the separability increases. To do. In addition, when determining whether or not the finger placement is appropriate by combining a plurality of image evaluations, it is not determined that “the finger is not placed” even though the finger is placed. It is desirable to set each threshold value closer to the image group “B”.

  The waveform processing unit 42 processes error information, waveform data, and the like, and particularly includes an error determination unit 42a, a peak detection unit 42b, and a pulse rate calculation unit 42c.

  The error determination unit 42a determines whether one or more of the luminance, edge amount, and center of gravity stored in the error information storage unit 32 are in a predetermined state. Further, threshold values used in the error determination unit 42a below are, for example, “Th_e1” (no finger is placed), “Th_e2” (poor finger placement is bad), “Th_yB2” (too bright), “ The determination criteria are “Th_yD2” (too dark), “Th_yB3” (bright), and “Th_yD3” (dark).

  Specifically, in the above-described example, when the image processing unit 41 performs image processing for a predetermined (plurality) “N1” frames or more, the error determination unit 42a has a cumulative evaluation score “ SErrP "is calculated.

  The error determination unit 42a compares “YMean” with a predetermined value “Th_yD2” when the number of frames subjected to image processing by the image processing unit 41 is less than a predetermined “N1” frame. Then, the error determination unit 42a determines that it is too dark when “YMean” is smaller than the predetermined value “Th_yD2”, and notifies the error display unit 21 of the abnormality. The error determination unit 42a ends normally when “YMean” is larger than the predetermined value “Th_yD2”.

  Then, when the calculated “SErrP” is smaller than the specified value “Th_e1”, the error determination unit 42 a determines that the finger is not placed and notifies the error display unit 21 of the abnormality. Subsequently, when “SErrP” is larger than the specified value “Th_e1”, the “SErrP” is compared with the specified value “Th_e2”. Thereafter, when “SErrP” is smaller than the prescribed value “Th_e2”, the error determination unit 42a determines that the finger is not placed and notifies the error display unit 21 of the abnormality.

  Then, the error determination unit 42a compares “YMean” with the specified value “Th_yB2” when “SErrP” is larger than the specified value “Th_e2”. Subsequently, the error determination unit 42 a determines that the brightness is too bright when “YMean” is larger than the specified value “Th_yB2”, and notifies the error display unit 21 of the abnormality.

  Thereafter, the error determination unit 42a compares “YMean” with the specified value “Th_yD2” when “YMean” is smaller than the specified value “Th_yB2”. Then, the error determination unit 42a determines that it is too dark when “YMean” is smaller than the specified value “Th_yD2”, and notifies the error display unit 21 of the abnormality.

  Subsequently, the error determination unit 42a compares “YMean” with the specified value “Th_yB3” when “YMean” is larger than the specified value “Th_yD2”. After that, the error determination unit 42a determines that it is bright when “YMean” is larger than the specified value “Th_yB3”, and notifies the error display unit 21 of a warning.

  Then, the error determining unit 42a compares “YMean” with the specified value “Th_yD3” when “YMean” is smaller than the specified value “Th_yB3”. Subsequently, when “YMean” is smaller than the specified value “Th_yD3”, the error determination unit 42a determines that the image is dark and notifies the error display unit 21 of a warning. The error determination unit 42a normally ends the process when “YMean” is larger than the specified value “Th_yD3”.

  Regarding the processing by the error determination unit 42a, the finger placement is not appropriate, and when an abnormality is notified, the processing is interrupted because the accuracy is poor even if a pulse wave is detected, and the finger placement is not appropriate. However, when the warning is notified, the pulse wave is detected without interrupting the processing.

  As for the processing order by the error determination unit 42a, the determination of “too bright” and “too dark”, the determination of “bright” and “dark” may be reversed, or is “too dark”? The determination of whether or not may be performed before the determination of how to place the finger. When it is not necessary to distinguish whether the image is bright because the way the finger is placed is bad or bright, or because the outside light is strong, whether the finger is too bright is determined before determining how to place the finger. It's also good.

  If an error is detected by the error determination unit 42a, the user is prompted to reposition the finger or move to a bright or dark place. Further, when a warning is issued by the error determination unit 42a, a warning to the user is issued. When no error is detected by the error determination unit 42a, the luminance information stored in the storage unit 30 is analyzed and the pulse rate is notified to the output unit 20 (see the processing of the pulse rate calculation unit 42c and the like). .

  In addition, as a setting method of each threshold value used in the error determination unit 42a, for example, when a finger is placed from a state darker than “Th_yD1” to a state exceeding “Th_yB1”, when the finger is placed in a bad way, And when a finger is not placed, a plurality of evaluation point accumulated data is obtained.

  Then, when the finger is placed, when the finger is placed in a bad way, and when the finger is not placed, the threshold values “Th_e2” and “Th_e1” are set. Also, the brightness at which the pulse wave cannot be taken is set to “Th_yB2” and “Th_yD2” in the data on which the finger is placed. Furthermore, among the data on which the finger is placed, the brightness at which the pulse wave is difficult to be taken is set to “Th_yB3” and “Th_yD3”. The threshold values “Th_yB3” and “Th_yD3” may be the same as the threshold values “Th_yD1” and “Th_yB1”.

  The peak detection unit 42 b detects a peak in the waveform of the image data from the luminance information of the image data stored in the luminance information storage unit 31. Specifically, in the example described above, the peak detection unit 42b calculates the local maximum value and the local minimum value from the time-series luminance information of the image data “G1” stored in the luminance information storage unit 31. Peaks (peak intervals) in the waveform of the image data “G1” are detected and stored in the peak information storage unit 33.

  The pulse rate calculation unit 42 c calculates the pulse rate based on the peak interval stored in the peak information storage unit 33. Specifically, the pulse rate calculation unit 42c calculates the pulse rate from the peak interval stored in the peak information storage unit 33, and outputs it to the pulse rate display unit 22, the file 23, and the like.

  As a simple example of calculating the pulse rate, when the frame rate is 15 FPS (Frame Per Second) and the average value of the peak intervals stored in the peak information storage unit 33 is 18, “18 ÷ 15 × 60 = 72 beats”. / Min ". In addition, it is good also as calculating a pulse rate after performing the process etc. which exclude a noise from the peak space | interval memorize | stored in the peak information storage part 33. FIG.

[Brightness average calculation processing]
Next, the average brightness calculation process according to the first embodiment will be described with reference to FIG. FIG. 7 is a flowchart for explaining luminance average calculation processing according to the first embodiment.

  As illustrated in FIG. 7, the pulse wave detection device 10 calculates the luminance sum “ttlY” of the image data “G1” (step S101). Then, the pulse wave detection device 10 calculates the luminance average “YMean” of the image data “G1” from the calculated luminance sum “ttlY” of the image data “G1” and stores it in the luminance information storage unit 31. (Step S102).

[Edge amount calculation processing]
Next, the edge amount calculation processing according to the first embodiment will be described with reference to FIG. FIG. 8 is a flowchart for explaining edge amount calculation processing according to the first embodiment.

  As shown in FIG. 8, the pulse wave detection device 10 calculates the horizontal edge “Edge_X” when flicker appears vertically in the image data “G1” (step S201 vertical direction) (step S202). Then, the pulse wave detection device 10 regards the calculated horizontal edge “Edge_X” as “0” that is equal to or less than the threshold (step S203), and calculates the total “SEEdge” of the edge amounts of the horizontal edge “Edge_X” (step S203). Step S204).

  Further, when the flicker appears in the horizontal direction in the image data “G1” (step S201 horizontal direction), the pulse wave detection device 10 calculates the vertical edge “Edge_Y” (step S205). Then, the pulse wave detection device 10 regards the calculated vertical edge “Edge_Y” as “0” that is equal to or lower than the threshold (step S206), and calculates the total edge amount “SEEdge” of the vertical edge “Edge_Y” (step S206). Step S207).

  The processes in step S203 and step S206 are processes for preventing noise from being picked up. Therefore, when the camera can sufficiently suppress noise, it is not necessary to make the process essential. Further, when flicker can be sufficiently suppressed, both the vertical and horizontal edges may be used and individually evaluated by the evaluation point calculation unit 41f, or the sum of the horizontal edge “Edge_X” and the vertical edge “Edge_Y”. Or the sum of squares may be evaluated.

[Luminance distribution calculation processing]
Next, the luminance distribution calculation process according to the first embodiment will be described with reference to FIG. FIG. 9 is a flowchart for explaining the luminance distribution calculation processing according to the first embodiment.

  As shown in FIG. 9, the pulse wave detection device 10 divides the image data “G1” into M × N regions (step S301). Then, the pulse wave detection device 10 calculates the luminance average Y (m, n) for each of the divided M × N regions (step S302).

  Subsequently, the pulse wave detection device 10 calculates the luminance average that is the average of all of the calculated luminance average Y (m, n), the maximum value “YMax”, the minimum value “YMin”, and Y (m, n). “YMean” is calculated (steps S303 and S304). Thereafter, the pulse wave detection device 10 calculates “Dst”, which is an evaluation value of the luminance distribution, from the calculated maximum values “YMax”, “YMin”, and “YMean” (step S305).

[Center of gravity evaluation value calculation processing]
Next, the center-of-gravity evaluation value calculation process according to the first embodiment will be described with reference to FIG. FIG. 10 is a flowchart for explaining the center-of-gravity evaluation value calculation processing according to the first embodiment.

  As shown in FIG. 10, the pulse wave detection device 10 calculates the sum of the horizontal luminances of the image data “G1” (step S401), and calculates the horizontal center of gravity position “Cx” (step S402). ). Then, the pulse wave detection device 10 calculates the sum of the vertical luminances of the image data “G1” (step S403), and calculates the vertical center of gravity position “Cy” (step S404).

  Subsequently, the pulse wave detection device 10 calculates the evaluation value “cPos” of the centroid of the image data “G1” from the calculated centroid positions “Cx” and “Cy” in the horizontal direction and the vertical direction (step) S405). In addition, as for the calculation order of the gravity center position in the horizontal direction and the vertical direction, either may be calculated first. Further, regarding the sum of the luminances in the horizontal direction and the vertical direction, when the difference between the maximum value and the minimum value is below a predetermined threshold value, the processing is performed by regarding “Cx” or “Cy” as “0”. It is good as well.

  Further, (a) in (2) of FIG. 2 seems to be a substantially uniform image, but actually, as shown in the image in which the intensity distribution shown in FIG. 11 is emphasized (the right diagram in FIG. 11). When the ambient light conditions and finger placement are appropriate, the center portion becomes bright and the surrounding becomes dark, so the centroid calculation unit 41e calculates an evaluation value for the centroid of the image data “G1”. . FIG. 11 is a diagram illustrating an example of an image having a luminance distribution.

[Evaluation score calculation process]
Next, the evaluation point calculation process according to the first embodiment will be described with reference to FIG. FIG. 12 is a flowchart for explaining an evaluation score calculation process according to the first embodiment.

  As illustrated in FIG. 12, the pulse wave detection device 10 compares the total edge amount “SEEdge” with the threshold value “Th_Edge1” (step S501), and when the total edge amount is larger than the threshold value (step S501). (Yes in S501), the evaluation score is “P1_1” (step S502).

  Then, when the total edge amount “SEdge” is smaller than the threshold “Th_Edge1” (No at Step S501), the pulse wave detection device 10 compares the luminance average “YMean” with the threshold “Th_yD1” (Step S503). ). Subsequently, when the average luminance is smaller than the threshold value (Yes at Step S503), the pulse wave detection device 10 sets the evaluation score “P1_2” (Step S504).

  Thereafter, when the luminance average “YMean” is larger than the threshold “Th_yD1” (No at Step S503), the pulse wave detection device 10 compares the luminance average “YMean” with the threshold “Th_yB1” (Step S505). Then, when the average luminance is smaller than the threshold (Yes at Step S505), the pulse wave detection device 10 sets the evaluation score “P1_3” (Step S506).

  Subsequently, when the luminance average “YMean” is larger than the threshold “Th_yB1” (No at Step S505), the pulse wave detection device 10 compares the evaluation value “Dst” of the luminance distribution with the threshold “Th_Dst1” ( Step S507). Thereafter, when the evaluation value of the luminance distribution is larger than the threshold value (Yes at Step S507), the pulse wave detection device 10 sets the evaluation point “P1_4” (Step S508).

  Then, when the evaluation value “Dst” of the luminance distribution is smaller than the threshold “Th_Dst1” (No at Step S507), the pulse wave detection device 10 compares the evaluation value “cPos” of the center of gravity with the threshold “Th_cPos1”. (Step S509). Subsequently, when the evaluation value of the center of gravity is larger than the threshold value (Yes at Step S509), the pulse wave detection device 10 sets the evaluation point “P1_5” (Step S510).

  After that, when the evaluation value “cPos” of the center of gravity is smaller than the threshold value “Th_cPos1” (No at Step S509), the pulse wave detection device 10 compares the total edge amount “SEEdge” with the threshold value “Th_Edge2” ( Step S511). Then, when the sum of the edge amounts is larger than the threshold value (Yes at Step S511), the pulse wave detection device 10 sets the evaluation score “P2_1” (Step S512).

  Subsequently, when the total edge amount “SEdge” is smaller than the threshold value “Th_Edge2” (No at Step S511), the pulse wave detection device 10 compares the luminance average “YMean” with the threshold value “Th_yD2” (Step S511). S513). Thereafter, when the average luminance is smaller than the threshold value (Yes at Step S513), the pulse wave detection device 10 sets the evaluation score “P2_2” (Step S514).

  Then, when the luminance average “YMean” is larger than the threshold “Th_yD2” (No at Step S513), the pulse wave detection device 10 compares the luminance average “YMean” with the threshold “Th_yB2” (Step S515). Subsequently, when the average luminance is smaller than the threshold (Yes at Step S515), the pulse wave detection device 10 sets the evaluation score “P2_3” (Step S516).

  Thereafter, when the luminance average “YMean” is larger than the threshold “Th_yB2” (No at Step S515), the pulse wave detection device 10 compares the evaluation value “Dst” of the luminance distribution with the threshold “Th_Dst2” (Step S515). S517). Then, when the evaluation value of the luminance distribution is larger than the threshold value (Yes at Step S517), the pulse wave detection device 10 sets the evaluation score “P2_4” (Step S518).

  Subsequently, when the evaluation value “Dst” of the luminance distribution is smaller than the threshold value “Th_Dst2” (No at Step S517), the pulse wave detection device 10 compares the evaluation value “cPos” of the center of gravity with the threshold value “Th_cPos2”. (Step S519). Thereafter, when the evaluation value of the center of gravity is larger than the threshold value (Yes at Step S519), the pulse wave detection device 10 sets the evaluation score “P2_5” (Step S520).

  Then, when the evaluation value “cPos” of the center of gravity is smaller than the threshold “Th_cPos2” (No at Step S519), the pulse wave detection device 10 sets the evaluation point “P_OK” (Step S521).

  In addition, about the determination of the said evaluation score, it determines in order of badness from evaluation score "P1", and when one evaluation item point was determined for processing load reduction, the process was complete | finished at that time, but all evaluation items It is good also as evaluating about and calculating a total point. However, if there is a negative point in one evaluation item, the negative point is often determined by affecting other evaluation items.

  Moreover, the order of evaluation items may be evaluated in any order, and the number of evaluation items is not limited to the above embodiment. Therefore, hereinafter, an example of determining evaluation points other than the above-described embodiment will be described with reference to FIGS.

(Evaluation of finger placement 1)
First, an example of determining an evaluation score based on how to place a finger will be described with reference to FIGS. 13 and 14. FIG. 13 is a diagram illustrating an example of evaluation point determination based on how to place a finger, and FIG. 14 is a flowchart for explaining processing in the evaluation point calculation unit 41f.

  For example, as shown in FIG. 13, when only a combination of brightness and edge amount is used, “0 point” is obtained when the brightness and edge amount are appropriate, “0” when dark or bright and the edge amount is large. The evaluation score is determined as “−2 points” when “−1 point”, too dark or too bright, and when the edge amount is too large.

  Next, the flow of processing when determining the evaluation points as shown in FIG. 13 will be described. As illustrated in FIG. 14, the evaluation score calculation unit 41f compares the luminance average “YMean” with the threshold “Th_yD1” (step S601), and when the luminance average is smaller than the threshold (Yes in step S601). The evaluation score is “−2” (step S602).

  Then, when the luminance average “YMean” is larger than the threshold value “Th_yD1” (No at Step S601), the evaluation score calculation unit 41f compares the luminance average “YMean” with the threshold value “Th_yB1” (Step S603). Subsequently, when the average luminance is larger than the threshold (Yes at Step S603), the evaluation score calculation unit 41f sets the evaluation score “−2” (Step S604).

  After that, the evaluation score calculation unit 41f is determined based on the sum of edge amounts “SEdge” and the luminance average “YMean” when the luminance average “YMean” is smaller than the threshold “Th_yB1” (No in Step S603). The threshold value “Th_Edge1 (YMean)” is compared (step S605). Then, the evaluation score calculation unit 41f sets the evaluation score “−2” (step S606) when the sum of the edge amounts is larger than the threshold value (Yes in step S605).

  Subsequently, the evaluation point calculation unit 41f determines that the brightness average “SEEdge” is smaller than the threshold “Th_Edge1 (YMean)” determined based on the brightness average “YMean” (No in step S605). “YMean” is compared with a threshold “Th_yD2” (step S607). Thereafter, when the average luminance is smaller than the threshold value (Yes at Step S607), the evaluation score calculation unit 41f sets the evaluation score “−1” (Step S608).

  Then, when the luminance average “YMean” is larger than the threshold value “Th_yD2” (No at Step S607), the evaluation score calculation unit 41f compares the luminance average “YMean” with the threshold value “Th_yB2” (Step S609). Subsequently, when the average luminance is larger than the threshold value (Yes at Step S609), the evaluation score calculation unit 41f sets the evaluation score “−1” (Step S610).

  Thereafter, when the luminance average “YMean” is smaller than the threshold value “Th_yB2” (No at Step S609), the evaluation score calculation unit 41f is determined based on the sum “SEdge” of the edge amounts and the luminance average “YMean”. The threshold value “Th_Edge2 (YMean)” is compared (step S611). Then, the evaluation score calculation unit 41f sets the evaluation score “−1” (step S612) when the sum of the edge amounts is larger than the threshold value (Yes in step S611). Note that the evaluation score calculation unit 41f ends the process with an evaluation score of “0” when the sum of the edge amounts is smaller than the threshold (No at Step S611) (Step S613).

  The present embodiment is not limited to the above description. For example, the threshold “Th_Edge1 (YMean)” in step S605 of FIG. 14 is set to the threshold “Th_Edge1 (ttlY) determined based on the total luminance“ ttlY ”. It is also possible to make a determination by replacing it with "." Further, for example, the threshold value “Th_Edge2 (YMean)” in step S611 in FIG. 14 may be replaced with a threshold value “Th_Edge2 (ttlY)” determined based on the total luminance “ttlY”. Further, for example, the luminance average “YMean” on the left side of Step S601, Step S603, Step S607, and Step S609 may be replaced with the luminance sum “ttlY” for determination.

(Evaluation point 2 for finger placement)
Next, with reference to FIG. 15 and FIG. 16, an example of evaluation point determination based on how to place a finger will be described. FIG. 15 is a diagram illustrating an example of evaluation points depending on how the finger is placed, and FIG. 16 is a flowchart for explaining processing in the evaluation point calculation unit 41f.

  For example, as shown in FIG. 15, when brightness is used to determine an evaluation point, “+1 point” is set when the finger placement is normal depending on the brightness, and “−1 point” is set when the finger placement is poor. The evaluation score is determined as “−2 points” when no finger is placed. When the center of gravity is used for determining the evaluation point, “+1 point” is set when the finger is placed normally according to the center of gravity, “0 point” is set when the finger is put in a bad way, and “−” is set when the finger is not put. An evaluation score is determined as “1 point”. When the edge amount is used to determine the evaluation point, “+1 point” when the finger placement is normal depending on the edge amount, “−1 point” when the finger placement is bad, and no finger is placed. Sometimes an evaluation score is determined as "-2 points".

  Next, the flow of processing when determining the evaluation points as shown in FIG. 15 will be described. As illustrated in FIG. 16, the evaluation score calculation unit 41f compares the luminance average “YMean” with the threshold “Th_yB1” (step S701), and when the luminance average is smaller than the threshold (Yes in step S701). The evaluation score is “−2” (step S702).

  Then, when the luminance average “YMean” is larger than the threshold value “Th_yB1” (No at Step S701), the evaluation score calculating unit 41f compares the total amount “SEEdge” of the edge amount with the threshold value “Th_Edge1” (Step S703). ). Subsequently, the evaluation score calculation unit 41f sets the evaluation score “−2” (step S704) when the sum of the edge amounts is larger than the threshold value (Yes in step S703).

  Thereafter, the evaluation point calculation unit 41f compares the evaluation value “cPos” of the center of gravity with the threshold value “Th_cPos1” when the sum “SEdge” of the edge amounts is smaller than the threshold value “Th_Edge1” (No in Step S703). Step S705). When the evaluation value of the center of gravity is larger than the threshold value (Yes at Step S705), the evaluation score calculation unit 41f sets the evaluation score “−1” (Step S706).

  Subsequently, when the evaluation value “cPos” of the center of gravity is smaller than the threshold value “Th_cPos1” (No at Step S705), the evaluation point calculation unit 41f compares the luminance average “YMean” with the threshold value “Th_yB2” (Step S705). S707). Thereafter, when the average luminance is smaller than the threshold value (Yes at Step S707), the evaluation score calculation unit 41f sets the evaluation score “−1” (Step S708).

  Then, when the luminance average “YMean” is larger than the threshold value “Th_yB2” (No at Step S707), the evaluation point calculation unit 41f compares the total amount “SEEdge” of the edge amount with the threshold value “Th_Edge2” (Step S709). ). Subsequently, when the sum of the edge amounts is larger than the threshold value (Yes at Step S709), the evaluation score calculation unit 41f sets the evaluation score “−1” (Step S710).

  After that, the evaluation point calculation unit 41f compares the evaluation value “cPos” of the center of gravity with the threshold value “Th_cPos2” when the total “SEEdge” of the edge amounts is smaller than the threshold value “Th_Edge2” (No at Step S709) ( Step S711). When the evaluation value of the center of gravity is larger than the threshold value (Yes at Step S711), the evaluation score calculation unit 41f sets the evaluation score “0” (Step S712). If the evaluation value of the center of gravity is smaller than the threshold value (No at Step S711), the evaluation score calculation unit 41f ends the process as an evaluation score “+1” (Step S713).

[Error judgment processing]
Next, an error determination process according to the first embodiment will be described with reference to FIG. FIG. 17 is a flowchart for explaining error determination processing according to the first embodiment.

  As shown in FIG. 17, the pulse wave detection device 10 calculates an accumulated value “SErrP” of evaluation points for the past “N2” frames when image processing is performed for a predetermined “N1” frame or more (Yes in step S801). (Step S802).

  The pulse wave detection device 10 compares “YMean” with a predetermined value “Th_yD2” when the number of frames subjected to image processing is less than the predetermined “N1” frame (No at Step S801) (Step S809). Then, when “YMean” is smaller than the predetermined value “Th_yD2” (Yes at Step S809), the pulse wave detection device 10 determines that the image is too dark and notifies the abnormality. The pulse wave detection device 10 ends normally when “YMean” is larger than the predetermined value “Th_yD2” (No at Step S809).

  When the calculated “SErrP” is smaller than the specified value “Th_e1” (Yes in step S803), the pulse wave detection device 10 determines that the finger is not placed and notifies the abnormality. Subsequently, when “SErrP” is larger than the prescribed value “Th_e1” (No at Step S803), the “SErrP” is compared with the prescribed value “Th_e2” (Step S804). After that, when “SErrP” is smaller than the prescribed value “Th_e2” (Yes in step S804), the pulse wave detection device 10 determines that the finger is not placed and notifies the abnormality.

  Then, when “SErrP” is larger than the specified value “Th_e2” (No at Step S804), the pulse wave detection device 10 compares “YMean” with the specified value “Th_yB2” (Step S805). Subsequently, when “YMean” is larger than the specified value “Th_yB2” (Yes at Step S805), the pulse wave detection device 10 determines that the brightness is too bright and notifies the abnormality.

  Thereafter, when “YMean” is smaller than the specified value “Th_yB2” (No at Step S805), the pulse wave detection device 10 compares “YMean” with the specified value “Th_yD2” (Step S806). Then, when “YMean” is smaller than the specified value “Th_yD2” (Yes at Step S806), the pulse wave detection device 10 determines that the image is too dark and notifies the abnormality.

  Subsequently, when “YMean” is larger than the specified value “Th_yD2” (No at Step S806), the pulse wave detection device 10 compares “YMean” with the specified value “Th_yB3” (Step S807). Thereafter, when “YMean” is larger than the specified value “Th_yB3” (Yes at Step S807), the pulse wave detection device 10 determines that the light is bright and notifies a warning.

  Then, when “YMean” is smaller than the specified value “Th_yB3” (No at Step S807), the pulse wave detection device 10 compares “YMean” with the specified value “Th_yD3” (Step S808). Subsequently, when “YMean” is smaller than the specified value “Th_yD3” (Yes at Step S808), the pulse wave detection device 10 determines that the pulse is dark and notifies a warning. The pulse wave detection device 10 normally ends the process when “YMean” is larger than the specified value “Th_yD3” (No at Step S808).

[Effects of Example 1]
As described above, the pulse wave detection device 10 according to the first embodiment appropriately determines how to place a finger photographed by the camera based on each characteristic of an image photographed by a portable terminal equipped with the camera. Therefore, it is possible to measure the pulse wave with high accuracy.

  For example, the pulse wave detection device 10 accepts an input image shot with a finger placed on the camera lens. Then, the pulse wave detection device 10 calculates a luminance average from the luminance sum of the input image, and calculates an edge amount in the horizontal direction or the vertical direction of the input image. Subsequently, the pulse wave detection device 10 calculates an evaluation value of the luminance distribution from the average luminance of the input image. Thereafter, the pulse wave detection device 10 calculates the position of the center of gravity in the horizontal direction and the vertical direction from the total luminance value in the vertical direction and the total luminance value in the horizontal direction of the input image, and based on the calculated center of gravity, An evaluation value is calculated. Then, the pulse wave detection device 10 calculates an evaluation score by comparing each of the calculated luminance average, edge amount, luminance distribution, and center of gravity with a predetermined threshold value. Subsequently, the pulse wave detection device 10 determines whether or not the finger is placed in a good state based on the calculated evaluation points, and the ambient light is in an appropriate state based on the luminance average. Determine whether. After that, the pulse wave detection device 10 outputs an error message when the finger is placed in a bad state or when the ambient light is not appropriate, and input when the finger is placed in a good state. Based on the signal obtained from the image, the pulse is calculated and output. As a result, the pulse wave detection device 10 can measure the pulse wave with high accuracy.

Although the embodiments of the present invention have been described so far, the present invention may be implemented in various different forms other than the embodiments described above. Accordingly, different embodiments will be described in (1) the configuration of the pulse wave detection device, (2) the order of evaluation score calculation processing, (3) values used for error determination, and (4) programs.

(1) Configuration of Pulse Wave Detection Device Information including the processing procedure, control procedure, specific name, various data and parameters shown in the above document and drawings (for example, the “error information storage unit 32 shown in FIG. 1) The information stored in “, etc.” can be arbitrarily changed unless otherwise specified.

  Further, each component of each illustrated apparatus is functionally conceptual, and does not necessarily need to be physically configured as illustrated. That is, the specific form of distribution / integration of each device is not limited to the illustrated one. For example, the waveform processing unit 42 having the error determination unit 42a is described, but the error determination unit 42a may be incorporated in the image processing unit 41. In addition, the error determination unit 42a is made independent as an “error processing unit”, and all or a part thereof is functionally or physically distributed / integrated in arbitrary units according to various burdens or usage conditions. can do. Furthermore, all or a part of each processing function performed in each device may be realized by a CPU and a program that is analyzed and executed by the CPU, or may be realized as hardware by wired logic.

(2) Order of Evaluation Point Calculation Processing In the first embodiment, the evaluation points are determined in the order of edge value, luminance average (dark), luminance average (bright), luminance distribution, and centroid evaluation value. However, the present invention is not limited to this, and the order can be changed as long as the score of the evaluation points to be determined is the same. For example, the pulse wave detection device 10 determines evaluation points in the order of luminance average (bright), edge amount, luminance average (dark), evaluation value of the center of gravity, and luminance distribution.

(3) Values used for error determination In the first embodiment, the error determination processing is described as being performed using evaluation points determined from the evaluation values of edge amount, luminance average, luminance distribution, and centroid. However, the present invention is not limited to this, and the error determination process can be performed using at least one (or a plurality) of the above characteristics. For example, the pulse wave detection device 10 performs the error determination process using a combination of the edge amount and the evaluation value of the center of gravity or a combination of the luminance distribution and the edge amount.

(4) Program In the above embodiment, the case where various processes are realized by hardware logic has been described. However, the present invention is not limited to this, and a program prepared in advance is executed by a computer. It may be realized by this. Therefore, in the following, an example of a computer that executes a pulse wave detection program having the same function as the pulse wave detection device 10 shown in the above embodiment will be described with reference to FIG. FIG. 18 is a diagram illustrating a computer that executes a pulse wave detection program.

  As shown in FIG. 18, a computer 110 serving as a pulse wave detection device is connected to an HDD 130, a CPU 140, a ROM 150, and a RAM 160 through a bus 180 or the like.

  The ROM 150 stores in advance a pulse wave detection program that exhibits the same function as the pulse wave detection device 10 described in the first embodiment, that is, a determination program 150a and an output program 150b as shown in FIG. ing. Note that these programs 150a to 150b may be appropriately integrated or distributed in the same manner as each component of the pulse wave detection device 10 shown in FIG.

  Then, the CPU 140 reads these programs 150a to 150b from the ROM 150 and executes them, so that the programs 150a to 150b function as a determination process 140a and an output process 140b as shown in FIG. The processes 140a to 140b correspond to the error determination unit 42a and the error display unit 21 shown in FIG.

  Then, CPU 140 executes a pulse wave detection program based on the data recorded in RAM 160.

  Note that the programs 150a to 150b are not necessarily stored in the ROM 150 from the beginning. For example, a flexible disk (FD), a CD-ROM, a DVD disk, a magneto-optical disk inserted into the computer 110, It is connected to the computer 110 via a “portable physical medium” such as an IC card, or a “fixed physical medium” such as an HDD provided inside or outside the computer 110, and further via a public line, the Internet, a LAN, a WAN, etc. Each program may be stored in “another computer (or server)” or the like, and the computer 110 may read and execute each program from now on.

1 is a diagram illustrating a configuration example of a pulse wave detection device according to Embodiment 1. FIG. It is a figure for demonstrating how to put a finger | toe and the characteristic of an image. It is a figure for demonstrating the point which determines the state of how to put a finger. It is a figure which shows the example of the image and waveform when how to put the finger put on the lens of a camera is bad. It is a figure which shows the example of illumination intensity around a camera. It is a figure which shows the brightness | luminance average example of the image corresponding to the illumination intensity around a camera. It is a figure which shows the example of a waveform when an image is bright. It is a figure which shows the example of a waveform when an image is dark. It is a figure which shows the example of a waveform in case an image is dark. It is a figure for demonstrating how to obtain | require the gravity center of an image. 7 is a flowchart for explaining a luminance average calculation process according to the first embodiment. 6 is a flowchart for explaining edge amount calculation processing according to the first embodiment; 6 is a flowchart for explaining luminance distribution calculation processing according to the first embodiment. 6 is a flowchart for explaining a center-of-gravity evaluation value calculation process according to the first embodiment. It is a figure which shows the example of the image with luminance distribution. 6 is a flowchart for explaining an evaluation score calculation process according to the first embodiment. It is a figure which shows the example of the evaluation point determination by how to put a finger. It is a flowchart for demonstrating the process in an evaluation score calculation part. It is a figure which shows the example of the evaluation score by how to put a finger. It is a flowchart for demonstrating the process in an evaluation score calculation part. 6 is a flowchart for explaining error determination processing according to the first embodiment; It is a figure which shows the computer which executes a pulse wave detection program.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Pulse wave detection apparatus 20 Output part 21 Error display part 22 Pulse rate display part 23 File 24 Pulsation display part 30 Storage part 31 Luminance information storage part 32 Error information storage part 33 Peak information storage part 40 Control part 41 Image processing part 41a Previous Processing unit 41b Luminance average calculation unit 41c Edge amount calculation unit 41d Luminance distribution calculation unit 41e Center of gravity calculation unit 41f Evaluation point calculation unit 42 Waveform processing unit 42a Error determination unit 42b Peak detection unit 42c Pulse rate calculation unit

Claims (3)

A pulse wave detection device that measures a pulse wave based on an image captured by an imaging means,
When the image is in a mode used for pulse acquisition, the state of the image is any one of luminance, an edge amount, and the center of gravity of the sum of luminance values in the vertical direction and horizontal direction of the image frame, or a plurality of them in a predetermined state. A determination unit for determining whether there is,
An output unit that outputs an error message when the determination result of the determination unit is not in a predetermined state, calculates a pulse based on a signal obtained from the image when the determination result is in a predetermined state, and outputs the calculation result; ,
A pulse wave detection device comprising: A pulse wave detection program for causing a computer to measure a pulse wave based on an image captured by an imaging means,
When the image is in a mode used for pulse acquisition, the state of the image is any one of luminance, an edge amount, and the center of gravity of the sum of luminance values in the vertical direction and horizontal direction of the image frame, or a plurality of them in a predetermined state. A determination procedure for determining whether there is,
An output procedure for outputting an error message when the determination result of the determination procedure is not in a predetermined state, calculating a pulse based on a signal obtained from the image when the determination result is in a predetermined state, and outputting the calculation result; ,
A pulse wave detection program for causing a computer to execute the above. A pulse wave detection method suitable for a pulse wave detection device that measures a pulse wave based on an image captured by an imaging means,
When the image is in a mode used for pulse acquisition, the state of the image is any one of luminance, an edge amount, and the center of gravity of the sum of luminance values in the vertical direction and horizontal direction of the image frame, or a plurality of them in a predetermined state. A determination step of determining whether there is,
An output step of outputting an error message when the determination result of the determination step is not in a predetermined state, calculating a pulse based on a signal obtained from the image when in a predetermined state, and outputting the calculation result; ,
A method for detecting a pulse wave, comprising:

Images