WO2023184832A1 - 生理状态检测方法、装置、电子设备、存储介质及程序 - Google Patents

生理状态检测方法、装置、电子设备、存储介质及程序 Download PDF

Info

Publication number
WO2023184832A1
WO2023184832A1 PCT/CN2022/113755 CN2022113755W WO2023184832A1 WO 2023184832 A1 WO2023184832 A1 WO 2023184832A1 CN 2022113755 W CN2022113755 W CN 2022113755W WO 2023184832 A1 WO2023184832 A1 WO 2023184832A1
Authority
WO
WIPO (PCT)
Prior art keywords
area
physiological state
information
smooth
target object
Prior art date
Application number
PCT/CN2022/113755
Other languages
English (en)
French (fr)
Inventor
何裕康
高勇
毛宁元
许亮
Original Assignee
上海商汤智能科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from CN202210344718.7A external-priority patent/CN114863399A/zh
Priority claimed from CN202210344726.1A external-priority patent/CN114648749A/zh
Priority claimed from CN202210346427.1A external-priority patent/CN114708225A/zh
Priority claimed from CN202210346417.8A external-priority patent/CN114663865A/zh
Application filed by 上海商汤智能科技有限公司 filed Critical 上海商汤智能科技有限公司
Publication of WO2023184832A1 publication Critical patent/WO2023184832A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/70—Multimodal biometrics, e.g. combining information from different biometric modalities

Definitions

  • the present disclosure relates to the field of computer technology, and relates to a physiological state detection method, device, electronic equipment, storage medium and program.
  • Accurate physiological status data is the basis for analyzing human body variability, so the detection of physiological status is of great significance.
  • effective physiological state detection can help understand the physiological state of the vehicle occupants, thereby providing auxiliary decisions for safe driving.
  • special detection equipment is mainly relied on, such as blood pressure monitors, heart rate monitors, oximeter and other equipment to detect physiological status.
  • smart watches, smart bracelets, etc. integrated with relevant sensing components can also be used. Wearable devices enable measurement of physiological states.
  • Embodiments of the present disclosure provide at least a physiological state detection method, device, electronic device, storage medium and program.
  • Embodiments of the present disclosure provide a physiological state detection method, which includes: acquiring a video stream collected by a camera device; extracting multiple frames of facial images of a target object from the multiple frame images in the video stream; determining the facial image At least one smooth region in the region; based on the region attribute information, perform weighting processing or region screening on the at least one smooth region to obtain the information to be extracted related to the physiological state; perform physiological state information extraction on the information to be extracted to obtain the information Describe the physiological status detection results of the target object.
  • Embodiments of the present disclosure also provide a physiological state detection device, including: an acquisition part configured to acquire a video stream collected by a camera device; an extraction part configured to extract a target object from multiple frames of images in the video stream A multi-frame facial image; a determining part configured to determine at least one smooth area in the facial image; a processing part configured to perform weighting processing or area filtering on the at least one smooth area based on area attribute information , obtain the information to be extracted related to the physiological state; the detection part is configured to extract physiological state information from the information to be extracted, and obtain the physiological state detection result of the target object.
  • An embodiment of the present disclosure also provides an electronic device, including: a processor, a memory, and a bus.
  • the memory stores machine-readable instructions executable by the processor.
  • the processor communicates with the The memories communicate through a bus, and when the machine-readable instructions are executed by the processor, the physiological state detection method as described above is performed.
  • Embodiments of the present disclosure also provide a computer-readable storage medium.
  • a computer program is stored on the computer-readable storage medium.
  • the computer program executes the physiological state detection method as described above when run by a processor.
  • Embodiments of the present disclosure also provide a computer program, the computer program including computer readable code, when the computer readable code is run in an electronic device, the processor of the electronic device executes to implement the The physiological state detection method described above.
  • the physiological state detection method, device, electronic device, storage medium and program provided by the embodiments of the present disclosure can extract multiple frames of facial images of the target object from the video stream and determine each frame At least one smooth area in the facial image, and then the determined at least one smooth area can be weighted or filtered based on the regional attribute information of the smooth area, so that the physiological characteristics of the target object can be obtained based on the weighted or filtered results.
  • Status detection results Compared with the inconvenient measurement problem caused by the need to use special instruments for contact measurement in related technologies, the present disclosure realizes physiological state detection based on image processing, and can perform real-time measurement anytime and anywhere, which is more practical. In this way, it can Improve measurement accuracy and flexibility.
  • Figure 1 shows a schematic flow chart of the first physiological state detection method provided by an embodiment of the present disclosure
  • Figure 2A shows a schematic diagram of facial feature points that can be extracted from facial images captured by a camera
  • Figure 2B shows a schematic flow chart of the second physiological state detection method provided by an embodiment of the present disclosure
  • Figure 2C shows a schematic flow chart of the third physiological state detection method provided by an embodiment of the present disclosure
  • Figure 2D shows a schematic flowchart of the fourth physiological state detection method provided by an embodiment of the present disclosure
  • Figure 2E shows a schematic flowchart of the fifth physiological state detection method provided by an embodiment of the present disclosure
  • Figure 3 shows a schematic diagram of a physiological state detection device provided by an embodiment of the present disclosure
  • FIG. 4 shows a schematic diagram of an electronic device provided by an embodiment of the present disclosure.
  • a and/or B can mean: A alone exists, A and B exist simultaneously, and B alone exists. situation.
  • at least one herein means any one of a plurality or any combination of at least two of a plurality, for example, including at least one of A, B, and C, which can mean including from A, Any one or more elements selected from the set composed of B and C.
  • rPPG remote photoplethysmographic
  • ROI Region of Interest
  • the picture may be too dark or too bright.
  • a single ROI area is often fixed to a face range as the ROI area. , it is easy to select an area with unsatisfactory exposure for signal extraction; or, in actual camera imaging, due to the angle between the face and the optical axis of the camera, the pixel area occupied by the ROI area in the image after imaging may be too small, resulting in ROI
  • the amount of regional information is small, making it difficult to obtain ideal detection results.
  • embodiments of the present disclosure provide a constraint through preset conditions, such as: selecting the optimal ROI area based on image brightness, or weighting based on area area, or weighting based on area area and area brightness, to achieve physiological status Detection scheme to obtain a signal quantity closer to the actual situation and improve detection accuracy.
  • the execution subject of the physiological state detection method provided by the embodiment of the present disclosure is generally an electronic device with certain computing capabilities.
  • the electronic device includes, for example: a terminal device or a server or other processing device.
  • the terminal device may be a user equipment (User Equipment, UE), a mobile device, a user terminal, a cellular phone, a cordless phone, or a personal digital assistant (Personal Digital Assistant, PDA). , handheld devices, vehicle-mounted devices, wearable devices, etc.
  • the physiological state detection method may be implemented by a processor calling computer readable instructions stored in a memory.
  • FIG. 1 there is a flow chart of a physiological state detection method provided by an embodiment of the present disclosure.
  • the method includes steps S101 to S105, wherein:
  • S102 Extract multi-frame facial images of the target object from the multi-frame images in the video stream;
  • S103 Determine at least one smooth area in the facial image
  • S104 Based on the regional attribute information, perform weighting processing or regional filtering on at least one smooth region to obtain information to be extracted related to the physiological state;
  • S105 Extract physiological state information from the information to be extracted, and obtain the physiological state detection results of the target object.
  • the physiological state detection method in the embodiment of the present disclosure can be applied to the automotive field where physiological state detection is required.
  • the embodiment of the present disclosure can realize the detection of the physiological state of the human body in the cabin environment.
  • the physiological state detection method provided by the embodiments of the present disclosure when the physiological state detection results of the target object in the cabin environment are obtained, it is possible to understand in a timely manner whether the target object has an abnormal physical condition, and to detect the abnormal physical condition of the target object when the physical condition appears. In abnormal situations, timely reminders or assistance can be provided to provide more realistic possibilities for safe driving.
  • the embodiments of the present disclosure can also be applied to any other related fields that require physiological state detection, such as medical treatment, home life, etc. Considering the wide range of applications in the field of automobile driving, examples will be given mostly in the automobile field.
  • the video stream in the embodiment of the present disclosure can be collected through a camera device (such as a fixed camera in the car in a car scene), or it can be collected by the user terminal's own camera, or it can be collected in other ways.
  • the installation position of the relevant camera can be preset based on the specific target object.
  • the camera in order to detect the physiological state of the driver in the vehicle, the camera here can be installed in the shooting location.
  • the scope covers the location of the driving area, such as the inside of the A-pillar of the vehicle, on the console, or the steering wheel position; for another example, in order to detect the physiological status of the occupants in the vehicle, including the driver and passengers, with various riding attributes , the cameras here can be installed on interior rearview mirrors, top trim, reading lights, etc.
  • the shooting range can cover multiple seating areas in the cabin.
  • the in-car image acquisition device included in the Driver Monitoring System can also be used to collect video streams related to the driving area, or the Occupant Monitoring System (Occupant Monitoring System) can be used , the in-car image collection device included in OMS realizes the collection of video streams related to the riding area.
  • DMS Driver Monitoring System
  • Occupant Monitoring System Occupant Monitoring System
  • facial blood vessels can reflect physiological states such as heartbeat and breathing
  • face detection on the multi-frame images in the video stream to extract multiple frames of the target object in the cabin. Facial image, and then extract physiological state information from the facial image.
  • the target object is a measurement object for human physiological state detection, which can be a human object staying at a specified location, or a human object with specific identity attributes.
  • the target object can be an object with specific ride attributes, such as the driver or the passenger in the passenger seat; or the target object can be an object whose identity has been registered using facial information in advance, such as through an application.
  • the registered car owner; alternatively, the target object can also be any occupant in the car. At least one occupant can be located through facial detection on the video stream in the cabin, and one or more detected occupants can be used as the target object.
  • the faces of multiple objects may appear on one frame of image.
  • the multi-frame face of the target object can be determined from the detected facial images. Image where the specified ride location is used to indicate the location of the target object being measured.
  • the relative position of the camera used to collect video streams in the vehicle cabin is fixed in the interior space of the vehicle.
  • the images collected can be divided according to the seating area.
  • a 5-seater private car can be divided into: driver's seat corresponding The image area corresponding to the passenger seat, the image area corresponding to the left rear seat, the image area corresponding to the right rear seat, and the image area corresponding to the middle rear seat.
  • the image area where the face of each passenger object falls can be determined, and then the passenger object in the designated riding position is determined as the target object. .
  • OMS In practical applications, OMS generally captures images of the entire cabin space, and may capture multiple people. You can manually select “front car seating space” or “rear seat seating space” to specify the cabin space to be measured. At this time, the embodiment of the present disclosure can measure the human face in the corresponding area in the image. DMS shoots for the main driving area. When the object captured by DMS only includes the driver, there is no need to specify the object.
  • physiological states such as heart rate, respiratory rate, blood oxygen, blood pressure, etc.
  • a continuous period of time in the video stream is used.
  • the image change information corresponding to multiple frames of facial images realizes the extraction of physiological state information, so that the extracted physiological state detection results are more in line with the needs of actual scenes.
  • the ROI area i.e., corresponding to the physiological state provided by the embodiments of the present disclosure
  • factors such as face rotation, occlusion, or external light.
  • the smooth area in the detection method is lost and detection cannot continue.
  • Embodiments of the present disclosure can use multiple smooth areas in the facial image for detection; in this way, even if one of the smooth areas is lost, the other smooth areas will perform well. In the case of detection, detection can also be continued, making the use scenarios more extensive.
  • the embodiments of the present disclosure can also select multiple smooth areas. And by weighting the pixel area of the smooth area in the picture, a signal amount closer to the actual situation can be obtained, which can improve the detection accuracy.
  • the smoothing area can be used to adjust the image in the picture.
  • the pixel area and corresponding pixel brightness are weighted to obtain a signal amount that is closer to the actual situation, which can improve detection accuracy.
  • At least one smoothing area can be determined for each frame of the facial image, and the smoothing area can be determined by screening one or more facial smoothing sub-regions corresponding to the facial image, wherein the facial smoothing sub-region can be Smooth connected area.
  • the connected area can be required to be a specified shape such as a rectangle, circle, or oval that can be located by facial key points.
  • Each connected area does not include the eyes, nose, mouth, etc. of the face. Non-smooth features such as eyebrows.
  • This connected area has a more uniform reflectivity to a certain extent, so that it can capture the changes in skin color and brightness caused by more effective facial blood vessel flow, and then achieve more accurate physiological status detection.
  • the physiological state detection method provided by the embodiment of the present disclosure can first determine at least one smooth region based on preset conditions related to image brightness settings, and then determine based on The image change information corresponding to the image information of at least one smooth area is used to extract physiological state information.
  • the physiological state detection results extracted here can be detection results including at least one of heart rate, respiratory rate, blood oxygen, blood pressure, etc. .
  • the optimal ROI area can be selected as the smoothing area by comparing the image brightness of multiple facial smoothing sub-areas, so that photoplethysmography (PPG) signals can be extracted for the smoothing area, thereby improving rPPG The detection accuracy of the method and the ability to resist interference during detection.
  • PPG photoplethysmography
  • the preset conditions set regarding the image brightness may be conditions for filtering out the relevant facial smoothing sub-regions that are obviously overexposed or obviously too dark. This is to take into account the abnormal image brightness corresponding to the obviously overexposed facial smoothing sub-regions. is high, the PPG signal extraction performed in this case is not accurate enough, which leads to the lower detection accuracy of the final physiological state detection result. Similarly, for the smooth sub-region of the face that is obviously too dark, the image Abnormally low brightness also leads to lower detection accuracy.
  • facial feature points can first be extracted from the facial image, and then at least one smooth area in the facial image can be determined based on the extracted facial feature points. That is, the above step S103 can be implemented through the following process:
  • Step 1 Extract facial feature points from the facial image to obtain the locations of multiple preset key feature points in the facial image;
  • Step 2 Based on the positions of the plurality of preset key feature points, determine at least one smooth area in the facial image corresponding to the plurality of preset key feature points.
  • facial feature points related to a standard face image can be preset.
  • the standard face here can include facial features.
  • the face image captured by the camera facing the camera; in this way, in the process of extracting facial feature points of the target object's facial image extracted from each frame of image, the extracted target object's facial image can be compared with the standard face Comparison between images to determine each facial feature point.
  • they can be eyebrow feature points, nose bridge feature points, nose tip feature points, cheek feature points, mouth corner feature points and other related feature points with obvious facial characteristics.
  • one or more smooth areas in the facial image may be determined based on the positions of multiple preset key feature points in the facial image.
  • the smooth area may be a rectangular area or other areas with a connected shape.
  • the embodiments of the present disclosure do not limit this. The following description will take a rectangular area as an example.
  • the smoothing area here can be a forehead smoothing area determined based on eyebrow feature points, a left upper cheek smoothing area and a right upper cheek smoothing area determined based on cheek feature points, nose bridge feature points and nose tip feature points, and The left lower cheek smoothing area and the right lower cheek smoothing area are determined based on the cheek feature points, nose tip feature points and mouth corner feature points.
  • the above five areas can be extracted simultaneously on one frame of face image.
  • the actual area that can be extracted from a frame of facial image can be determined according to the actual situation.
  • the following steps may be followed to determine the occluded smooth area:
  • Step 1 When the preset key feature points are missing from the extracted facial feature points, determine the facial sub-region corresponding to the missing preset key feature point as the missing region, and determine the predefined multiple facial sub-regions. Other facial sub-regions in the region except the missing region are non-missing regions;
  • Step 2 Determine the boundary of the non-missing area based on the position of the preset key feature point corresponding to the non-missing area among the extracted facial feature points to determine at least one smooth area.
  • the preset key feature points are missing from the extracted facial feature points, indicating that the corresponding facial sub-region is occluded, and the facial sub-region corresponding to other facial key points that are not missing can be determined.
  • the smooth area is determined. It can be seen that even if occlusion occurs, the embodiment of the present disclosure can extract the smooth area more accurately according to the above method.
  • FIG. 2A it is a schematic diagram of the facial feature points that can be extracted from the facial image captured by the camera; wherein, a total of 106 feature points are extracted from the face.
  • 5 smooth areas can be screened, as shown in Figure 2A, where area 1 can be a rectangular ROI of area 1 constructed by two feature points of the eyebrows on both sides; area 2 is the left A cheek area on the right side, the rectangular ROI of area 2 can be constructed through the positions of the left edge feature points of the face, the nose bridge feature points, and the left eye feature points; Area 3 is a cheek area on the right side, and can be constructed through the positions of the right edge of the face The positions of the feature points, nose bridge feature points, and right eye feature points construct the rectangular ROI of area 3.
  • Area 4 is another cheek area on the left side. It can be used through the left edge feature points of the face, the left nose feature points, and the left mouth corner features. The position of the points constructs the rectangular ROI of area 4.
  • Area 5 is another cheek area on the right side. The rectangular ROI of area 5 can be constructed through the positions of the right edge feature points of the face, the right nose feature points, and the right corner of the mouth feature points.
  • facial pose of the target object can be detected based on the facial image, and then facial feature points are extracted from the facial image based on the facial pose.
  • the above facial pose may be determined based on a pre-trained facial pose detection network.
  • the facial pose detection network can be a neural network of any structure, such as a multi-layer convolutional neural network, which can learn the correspondence between facial image samples and corresponding annotated facial poses.
  • the facial poses that can be annotated here include head The rotation direction, rotation angle and other information of the head relative to the camera.
  • the facial pose of the face can be determined.
  • the face can be tilted 45° to the left.
  • the face can be tilted to the left.
  • the preset key feature points are extracted based on the standard left-skewed face image. The operation is simple.
  • the areas of different facial sub-areas presented in the image may be different for different facial postures, and the areas of the corresponding smooth areas may also be different. Taking the face tilted 45° to the left as an example, At this time, the area of the left cheek area is very small or even almost invisible in the image, and the area of the corresponding smooth area is close to 0.
  • the above-mentioned area attribute information may be information related to at least one attribute of the brightness, area, location, etc. of the smooth area.
  • the weight of each smooth area can be calculated based on the above-mentioned area attributes of each smooth area, or a screening process can be performed. After the weighting or screening process, the image information of the smooth area related to the physiological state is obtained as the information to be extracted. ; Or, after calculating the weights, perform a time domain signal extraction operation, and weight the time domain signals corresponding to each smooth area to obtain the signal as the information to be extracted related to the physiological state.
  • the information to be extracted related to the physiological state includes image information of at least one region of interest.
  • smoothing of at least one region based on regional brightness can be achieved.
  • Regions are screened to obtain at least one region of interest, and then physiological state information can be extracted from the image information of at least one region of interest to obtain physiological state detection results, as shown in Figure 2B, which is the third method provided by the embodiment of the present disclosure.
  • Figure 2B is the third method provided by the embodiment of the present disclosure.
  • Schematic flow chart of two physiological state detection methods. Step S104 provided by the embodiment of the present disclosure can be implemented through the following step S201:
  • S201 Determine at least one region of interest whose brightness meets a preset condition from at least one smooth region.
  • S105 provided in the above embodiment can be implemented through the following step S202:
  • S202 Extract physiological state information from the image information of at least one area of interest to obtain physiological state detection results.
  • filtering of areas of interest based on image brightness thresholds can be achieved through the following steps:
  • Step 1 Determine the image brightness threshold corresponding to the face image and the regional image brightness of each smooth area
  • Step 2 Filter at least one smooth area according to the image brightness threshold and the regional image brightness to obtain at least one candidate smooth area
  • Step 3 From at least one candidate smoothing area, select at least one area of interest whose area brightness meets the preset brightness condition.
  • the smooth area can first be filtered based on the image brightness threshold, and then the area of interest with the best brightness can be selected based on the candidate smooth areas obtained by filtering.
  • embodiments of the present disclosure can be defined by the image brightness threshold of maximum brightness, or by the image brightness threshold of minimum brightness.
  • the former can filter out smooth areas where the brightness of the regional image exceeds a first preset ratio of the maximum brightness. For example, smooth areas where the brightness of the regional image exceeds 90% of the maximum brightness can be filtered out to filter out related smooth areas that may be overexposed. ;
  • the latter can filter out smooth areas where the brightness of the regional image does not reach the second preset ratio of the minimum brightness. For example, smooth areas where the brightness of the regional image is less than 30% of the minimum brightness can be filtered out to filter out the smooth areas that may be too dark.
  • Related smooth areas can be defined by the image brightness threshold of maximum brightness, or by the image brightness threshold of minimum brightness.
  • the regional image brightness mentioned above is a numerical value; wherein, the numerical value can refer to the average value of the regional brightness, that is, the value obtained by adding the pixel brightness values of all pixels in the region and taking the average, or it can also refer to the average value within the region.
  • the maximum value of pixel brightness among all pixels can also refer to the minimum value of pixel brightness among all pixels in the area, or it can be the value obtained by calculating the pixel brightness values of all pixels in the area according to a preset calculation method.
  • the maximum brightness and the minimum brightness can be preset brightness values, and different brightness values can be set for different application scenarios and/or different image formats. In actual applications, they can be set as needed. to adjust. Here, considering that within a certain brightness range, areas with higher brightness are more likely to have clearer image features, so that the physiological detection results obtained will be more accurate. Based on this, in the embodiments of the present disclosure, Select the candidate smooth area with the maximum brightness as the smooth area.
  • the brightness weight of each smooth area can be determined first, and then the physiological state information extraction results of the multiple smooth areas are fused based on the brightness weight.
  • the relevant brightness weight may be determined based on the regional image brightness of the corresponding region. For example, a smooth region with a greater regional image brightness may correspond to a larger brightness weight, and conversely, a smooth region with a smaller regional image brightness may correspond to a smooth region with a smaller regional image brightness. Smaller brightness weights can highlight the impact of smooth areas with larger brightness on physiological state detection, while weakening the impact of smooth areas with smaller brightness on physiological state detection. In this way, the physiological state detection results obtained by fusion will be more accurate. .
  • Physiological state information is extracted from the image information of the area.
  • the image information of the area of interest includes pixel brightness information of three different color channels.
  • physiological state information is extracted from the image information of at least one area of interest to obtain the physiological state of the target object.
  • the status detection result that is, the above step S202, can be achieved through the following steps:
  • Step 1 Based on the pixel brightness information of the three color channels of the area of interest, determine the time domain signal that represents the physiological state of the target object;
  • Step 2 Convert the time domain signal to frequency domain to obtain a frequency domain signal that represents the physiological state of the target object;
  • Step 3 Determine the physiological state value of the target object based on the peak value of the frequency domain signal.
  • the physiological state directly affects the blood flow changes of the target object, and the blood flow changes in turn affect the brightness changes of the image
  • the area of interest corresponds to red, green, and blue.
  • the time-domain brightness signal of each of the three color channels forms an RGB three-dimensional signal.
  • the time-domain brightness signals of the three different color channels are subjected to principal component analysis to extract the principal components (dimensionality reduction) to obtain the one-dimensional
  • the signal is a time-domain signal that represents the physiological state of the target object.
  • the time-domain signal can be determined by the time-domain brightness signal of one of the above-mentioned color channels (for example, the green channel), and the selected channel can be the one that best One channel that represents changes in blood flow, that is, the above step 1 regarding "determining the time domain signal that represents the physiological state of the target object based on the pixel brightness information of the three color channels of the area of interest" can be achieved through the following process:
  • the time domain brightness signal corresponding to each color channel of the smooth area is determined.
  • image change information corresponding to multiple frames of facial images in the video stream for a period of time is required, which This is because physiological status information such as heart rate, respiratory rate, blood oxygen, and blood pressure often require a certain amount of time to be detected, and in the process of determining the time domain brightness signal corresponding to the area of interest, the brightness values of multi-color channels can be used to Improve the accuracy of time domain brightness signal extraction.
  • processing such as regularization and Detrend filtering denoising can be performed before performing principal component analysis on the three-dimensional time domain brightness signal.
  • the obtained time domain signal can also be denoised by sliding average filtering, which can improve the accuracy of the time domain signal and improve the accuracy of the physiological state detection results obtained through subsequent processing.
  • the pixel areas occupied by different smooth areas in the camera imaging are different, it is possible to select at least one smooth area and By weighting the pixel area of the smooth area in the picture, a signal amount closer to the actual situation is obtained and the detection accuracy is improved.
  • the physiological state detection method can first determine the time domain brightness signal corresponding to each smooth area, and then calculate each time domain brightness signal based on the area of each smooth area. Weighting processing is performed to facilitate the extraction of physiological state information based on the area-weighted time domain brightness signal.
  • the physiological state detection results extracted here can be detection including at least one of heart rate, respiratory rate, blood oxygen, blood pressure, etc. result. Then, when the corresponding time-domain brightness signal is determined for each smooth area, the determined time-domain brightness signal can be weighted based on the area of the smooth area.
  • the area-weighted time-domain brightness signal can highlight the face to a greater extent.
  • the information to be extracted related to the physiological state may include a weighted time-domain brightness signal.
  • smoothing of at least one The time-domain brightness signal corresponding to the area is weighted to obtain an area-weighted time-domain brightness signal.
  • physiological state information can be extracted based on the area-weighted time-domain brightness signal to obtain the physiological state detection result, as shown in Figure 2C.
  • S104 provided by the embodiment of the present disclosure can be implemented through the following step 203 and step S204:
  • S203 For each smooth area, generate a time domain brightness signal corresponding to the smooth area based on the pixel brightness information of at least one color channel of the smooth area in the multi-frame facial image;
  • S105 provided in the above embodiment can be implemented through the following step S205:
  • S205 Extract physiological state information based on the area-weighted time domain brightness signal to obtain physiological state detection results.
  • step S203 provided by the embodiment of the present disclosure can be implemented through the following process::
  • step S204 provided in the above embodiment can be implemented through the following process:
  • the time domain brightness signal of at least one smooth area under the color channel is weighted based on the area of each smooth area to obtain an area-weighted time domain brightness signal under the color channel.
  • step S205 provided in the above embodiment can be implemented through the following process:
  • Step 1 Perform principal component analysis on the area-weighted time-domain brightness signal of the smooth area under multiple different color channels to obtain a time-domain signal that represents the physiological state of the target object;
  • Step 2 Perform frequency domain processing on the time domain signal representing the physiological state of the target object, and determine the physiological state detection result based on the peak value of the frequency domain signal obtained by the frequency domain processing.
  • the area weight of the corresponding smooth area can be determined based on the area of each smooth area, and the area weight is used to determine the area weight of the corresponding smooth area.
  • the time-domain brightness signal corresponding to at least one smooth area is weighted to obtain a signal quantity closer to the actual situation, thereby improving the accuracy of the generated physiological state detection results.
  • different facial postures present different facial sub-regions in the image with different area sizes, and the corresponding smooth areas are also different.
  • the face is still tilted 45° to the left.
  • the left cheek area is almost invisible in the image, and the corresponding smooth area has an area of 0.
  • the area of each smooth area may also be estimated based on facial posture and parameters of the camera device. For example, after facial pose detection is completed through neural networks, etc., the face model facing the camera is projected into the image coordinate system according to parameters such as facial pose angle and camera focal length, and each smoothing function based on facial key point positioning is calculated. The boundary point coordinates of the area in the image coordinate system are used to estimate the area value of each smooth area in the image. When some areas of the face are not bright enough or some key points are missing, this area value can be used to estimate the area of each smooth area in the image. In the embodiment of the present disclosure, the area of the smooth area can be determined based on the position of the preset key feature points corresponding to each smooth area.
  • the position of each preset key feature point can be used as the vertex position of the smooth area, and the length and width of the smooth area are determined by the distance between the vertices, and then the area of the area is determined, so that the area of the area determined is more accurate.
  • Embodiments of the present disclosure can determine the area weight corresponding to each smooth area based on the proportion of each area area to the total area area, and then perform area weighting.
  • the threshold may be a preset fixed value, or a value determined based on the area distribution of each smooth area in the entire face area in the current image. For example, a certain percentage (for example, 20%) of the maximum value of the area of each smooth region in the current image can be used as the threshold.
  • the smooth area in the image whose visible range does not meet the preset visibility requirements can be removed from the smooth area based on the facial posture. Taking the face still tilted 45° to the left as an example, the area of the right cheek area is less than The preset threshold is almost invisible in the image, and the corresponding smooth area can be directly removed.
  • the information to be extracted related to the physiological state includes fusion Image information.
  • the fused image information here represents image information that fuses multiple dimensions of image factors.
  • the first contribution of each smooth area to the physiological state detection result and the third contribution of each smooth area to the physiological state detection result can be determined first. Two contribution degrees, where the first contribution degree represents the contribution of the image information related to the area of each smooth area to the physiological state detection results, and the second contribution degree represents the contribution of the image information related to the area brightness in each smooth area to the physiological state detection.
  • the degree of contribution of the result, and then the image information of at least one smooth area is fused based on the first contribution degree and the second contribution degree to obtain the fused image information, and finally the physiological state information is extracted based on the fused image information.
  • the physiological state extracted here is detected
  • the result may be a detection result including at least one of heart rate, respiratory rate, blood oxygen, blood pressure, etc.; in this way, the first priority of each smooth area to the physiological state detection result can be determined based on the area and pixel brightness information of each smooth area.
  • the contribution degree and the second contribution degree the image information of at least one smooth area can be fused, and finally the physiological state information is extracted based on the image information obtained by the fusion, and the physiological state detection result is obtained.
  • FIG. 2D it is a schematic flow chart of the fourth physiological state detection method provided by an embodiment of the present disclosure, that is, step S104 provided by the above embodiment, which can also be implemented through the following steps S206 to step S208:
  • S206 Determine the first contribution of each smooth area to the physiological state detection result according to the area of each smooth area
  • S208 Based on the first contribution degree and the second contribution degree, fuse the image information of at least one smooth area to obtain fused image information.
  • S105 provided in the above embodiment can be implemented through the following step S209:
  • S209 Extract physiological state information from the fused image information to obtain physiological state detection results.
  • the relevant fusion process may be weighted fusion based on the first contribution degree and the second contribution degree. That is, for a smooth area, the higher the corresponding first contribution degree and the second contribution degree, the higher the corresponding first contribution degree and the second contribution degree. The representation ability of the fused image information corresponding to the smooth area is stronger.
  • the above fusion process may also be performed based on the comparison result of the first contribution degree and the second contribution degree.
  • the physiological state information extraction result corresponding to the smooth area can be determined based only on the area of the smooth area, ignoring the influence of brightness.
  • the first contribution can be determined based on the area of the corresponding smooth area, and the first contribution value is proportional to the area of the corresponding smooth area. That is, the smooth area with a larger area will be used for physiological state detection to a certain extent. Provide a higher first contribution. On the contrary, the smaller the area, the smoother area will provide a lower first contribution to physiological state detection. This is considering that as the area of the area increases, the effective information that can be extracted is The amount of information will also increase accordingly, and the increase in the amount of effective information can improve the accuracy of physiological state detection to a certain extent.
  • the second contribution degree may be determined based on the pixel brightness information of the corresponding smooth area.
  • the second contribution degree is proportional to the average pixel brightness value of the corresponding smooth area. That is, the smooth area with stronger pixel brightness will be to a certain extent. Provide a higher second contribution to physiological state detection. On the contrary, the smoother area with weaker pixel brightness will provide a lower second contribution to physiological state detection. This is considering that as the pixel brightness increases, the corresponding The image quality will also be improved accordingly, and better image quality can also improve the accuracy of physiological state detection to a certain extent.
  • the second contribution is inversely proportional to the variance of the pixel brightness value of the corresponding smooth area. In this case, the corresponding second contribution can be determined based on the variance value.
  • the second contribution degree is determined to ensure the accuracy of the final physiological state detection result.
  • the first contribution corresponding to each smooth area can be determined based on the area of each smooth area; on the other hand, the first contribution degree corresponding to each smooth area can be determined based on the area of each smooth area.
  • the pixel brightness information determines the second contribution corresponding to each smooth area.
  • the first contribution of the smooth region is determined to be 0. That is, for a smaller smooth region, its image representation ability is limited, and the corresponding value may not be considered here.
  • the first contribution of The contribution degree is 0, or when the average brightness value is greater than the second preset brightness threshold, the second contribution degree of the smooth area is determined to be 0, that is, for smooth areas that may be overexposed or too dark, they may not be considered here.
  • the corresponding second contribution degree is determined to be 0.
  • the area weight can be implemented according to the following steps:
  • Step 1 Sum the areas of each smooth area to obtain the area sum
  • Step 2 For each smooth area, calculate the ratio of the area of the smooth area to the sum of the areas, and obtain the area ratio of the smooth area as the area weight corresponding to the smooth area.
  • the area sum value can be obtained.
  • a greater area weight can be determined.
  • a larger area weight can be determined. Smooth regions whose area accounts for less of the area and value determine a smaller area weight.
  • the brightness weight can be implemented as follows:
  • Step 1 Sum the average brightness values of each smoothed area to obtain the sum of brightness values
  • Step 2 For each smooth area, calculate the ratio of the average brightness value of the smooth area to the sum of brightness values, and obtain the brightness ratio of the smooth area as the brightness weight corresponding to the smooth area.
  • the brightness sum value can be obtained.
  • a greater brightness weight can be determined, and vice versa.
  • a smaller brightness weight can be determined for smooth areas where the brightness of the smooth area accounts for less of the brightness sum value.
  • the image information of each smooth area can be fused based on the area weight and the brightness weight for subsequent physiological state detection, where the first contribution includes the area weight and the second contribution includes the brightness weight.
  • the fusion process that is, the above-mentioned step S208, can be implemented through the following process:
  • Step 1 Determine the total weight value of each smooth area based on the product of the area weight and brightness weight corresponding to each smooth area;
  • Step 2 Perform weighted fusion on the image information of at least one smooth area based on the corresponding total weight value to determine the fused image information.
  • the total weight value corresponding to each smooth area can be determined first, and then, when the image information of each smooth area is assigned a corresponding total weight value, the fused image information can be determined through weighted summation.
  • the corresponding total weight value is also larger, which can provide more information support for fused image information to a certain extent, while for area A smooth area with a large weight and a small brightness weight, or a smooth area with a small area weight and a large brightness weight, may have a corresponding total weight value that may be larger or smaller, which needs to be determined based on the image analysis results.
  • the fused image information obtained through weighted fusion can mine effective pixels on the face to the greatest extent, which provides more data support for subsequent physiological state detection, thereby helping to improve detection accuracy.
  • the embodiment of the present disclosure when the first contribution degree and the second contribution degree of each smooth area to the physiological state detection result are determined, and the fused image information includes the fused time domain brightness signal, the embodiment of the present disclosure
  • the provided physiological state detection method can first determine the time domain brightness signal corresponding to each smooth area through time domain processing, and then fuse the time domain brightness signals based on the first contribution and the second contribution of each smooth area, thereby facilitating Physiological state information extraction based on fused time-domain brightness signals.
  • frequency domain processing can be performed on the fused time domain brightness signal. More useful information can be analyzed based on the frequency domain signal obtained by frequency domain processing, for example, The amplitude distribution and energy distribution of each frequency component can be determined, thereby obtaining the frequency values of the key amplitude and energy distribution.
  • the physiological state value of the target object can be determined based on the peak value of the frequency domain signal.
  • the time domain brightness signal corresponding to each color channel may be determined based on the brightness values of the three color channels corresponding to the fused image information.
  • Step 1 For each smooth area, perform time domain processing on the brightness value of the smooth area in the multi-frame image to determine the time domain brightness signal corresponding to the smooth area;
  • Step 2 Based on the first contribution degree and the second contribution degree, fuse the time domain brightness signals corresponding to the multiple smooth areas to obtain a fused time domain brightness signal.
  • step S209 provided in the above embodiment can be implemented through the following process:
  • the fused time domain brightness signal is subjected to frequency domain processing, and the physiological state result is determined based on the peak value of the frequency domain signal obtained by the frequency domain processing.
  • performing frequency domain processing on the fused time domain brightness signal may refer to performing principal component analysis on the fused time domain brightness signal to obtain the corresponding time domain signal, and then performing frequency domain processing on the time domain signal. After conversion, a frequency domain signal representing the physiological state of the target object is obtained; finally, based on the peak value of the frequency domain signal, the physiological state result of the target object is determined.
  • the facial image of the target object when a video stream is obtained, can be extracted from the video stream first, and then the body profile data of the target object can be searched based on the facial image.
  • the image information of the body image is processed in time domain and frequency domain to obtain the heart rate estimate of the target object.
  • the blood pressure value of the target object can be estimated and the blood pressure measurement is obtained. result.
  • the physiological state detection method can determine the heart rate estimate used to characterize blood pressure fluctuations, and then combine it with the body
  • the file data enables accurate measurement of blood pressure values, and the measurement results are more accurate. Without the need for professional equipment, real-time measurement can be performed anytime and anywhere, making it more practical.
  • blood pressure refers to the lateral pressure acting on the blood vessel wall per unit area when the blood flows in the blood vessel. It is the driving force that promotes the flow of blood in the blood vessel and is a very important reflection of human health in daily life. physiological indicators.
  • the blood will flow to the whole body along the aorta, and the pumped blood is the measured blood pressure.
  • the level of blood pressure is related to the power of the heart to pump blood. That is to say, there is a certain connection between the human body's heartbeat and human blood pressure.
  • the image change information corresponding to multiple frames of facial images in the video stream that lasts for a period of time can be used to determine the heart rate estimate, and then combined with Body profile data is used to determine blood pressure measurement results, so that the measured blood pressure value can be combined with heartbeat conditions and specific attributes of different human bodies, thus more in line with the needs of actual scenarios.
  • the physiological state detection result of the target image includes a blood pressure measurement result
  • the above embodiment may perform the following steps S210 to S212, as shown in FIG. 2E, provided by the embodiment of the present disclosure.
  • S210 Based on the facial image, search for body profile data matching the target object from the preset archive;
  • S211 Perform time domain processing and frequency domain processing based on the image information of the facial image in multiple frames to obtain the heart rate estimate of the target object;
  • S212 Based on the body profile data and the heart rate estimate, estimate the blood pressure value of the target object and obtain the blood pressure measurement result.
  • the relevant body profile data may include data such as height, weight, gender data, age, etc. This is considering that human bodies with different heights, weights, gender data, and ages have different body functions, for example Children's heartbeats are generally slower than those of the elderly.
  • the purpose of estimating blood pressure values by combining body profile data and heart rate estimates here is to achieve blood pressure measurement that is suitable for all types of human bodies and is more adaptable.
  • step S210 can be implemented in at least one of the following ways:
  • Method 1 Based on the facial features of the target object extracted from the facial image, the body profile data corresponding to the facial features of the target object is searched from the preset archive library;
  • Method 2 Based on the identification of the target object recognized by the facial image, the body profile data corresponding to the identification is searched from the preset archives.
  • the body profile data matching the target object can be searched from the preset archive based on the facial image. That is, the body profile data of each object can be saved in the preset archive in advance. Once the face is obtained, Based on the facial image or the facial features extracted from the facial image, the body profile data corresponding to the target object can be searched from the preset archive, and the operation is simple. Alternatively, some body profile data can also be obtained by analyzing the target object's facial image, such as age, gender, etc.
  • facial features of the target object can be extracted based on the facial image, and then body profile data corresponding to the facial features of the target object are searched from a preset archive.
  • the relevant facial features can be related features including the distance between the eyes, the distance between the eyebrows, etc., or they can be obtained by extracting features from human face images through deep learning models such as convolutional neural networks for recognizing faces.
  • feature representation when the facial features of each object can be pre-recorded in the preset archive, based on the similarity between the extracted facial features and each facial feature in the preset archive, it can be Find the body profile data corresponding to the target object.
  • the target object can be identified based on the facial image to determine the identity of the target object, and then the body profile data corresponding to the identity is searched from the preset archives as the body profile data matching the target object. . That is, different identifiers can be set for each object in advance, and in the case of face comparison based on facial images, the target object corresponding to the identifier and its body profile data can be determined.
  • the heart rate estimation value can be obtained by performing time domain processing and frequency domain processing based on the extracted image information of multiple frames of facial images. In practical applications, it can be based on multiple color channels.
  • the time domain processing and frequency domain processing of image information can obtain a heart rate estimate that is more in line with actual needs.
  • the implementation process can refer to the image information value based on the smooth area to extract physiological state information to obtain the physiological state detection results of the target object, where, The physiological state detection result is the heart rate estimate.
  • the process of achieving the blood pressure measurement result of the target object that is, the above-mentioned step S212, can be implemented through the following steps:
  • Step 1 Determine the ejection time corresponding to the estimated heart rate value based on the linear fitting relationship between ejection time and heart rate; and, based on the gender data, height and weight included in the body profile data, calculate the human body surface area corresponding to the target object; and , based on the weight and age included in the body profile data, and the heart rate estimate of the target object, determine the heart elasticity value of the target object;
  • Step 2 Determine the stroke volume corresponding to the target object based on the ejection time, human body surface area, age included in the body profile data, and heart rate estimate;
  • Step 3 Calculate the pulse pressure estimate corresponding to the target object based on the ratio between the stroke volume corresponding to the target object and the cardiac elasticity value;
  • Step 4 Calculate the systolic blood pressure value and diastolic blood pressure value of the target subject based on the pulse pressure estimate corresponding to the target subject and the gender data included in the body profile data.
  • blood pressure measurements including systolic blood pressure values and diastolic blood pressure values may be achieved in conjunction with the subject's individual body profile data and the estimated heart rate estimates.
  • the following are the relevant professional terms in the blood pressure measurement results:
  • Systolic blood pressure When the heart contracts to pump blood from the ventricles into the arteries, the pressure exerted by the blood flow on the arterial walls is the systolic blood pressure.
  • Diastolic blood pressure When the heart relaxes, the pressure at which the arterial wall has a certain elasticity and continues to push the blood flow forward is called diastolic blood pressure.
  • Pulse pressure The difference between systolic and diastolic blood pressure.
  • Mean arterial pressure The average arterial blood pressure during a cardiac cycle.
  • Stroke Volume refers to the amount of blood ejected from one ventricle in one heart stroke.
  • Ejection time Left ventricular ejection time (LVET), which is defined as the time interval from the opening of the aortic valve to the closing of the aortic valve. It is the systolic period during which the left ventricle ejects blood into the aorta. Abbreviated as (Ejection Time, ET).
  • LVET Left ventricular ejection time
  • Xu Shengwen's formula is a human body surface area formula that conforms to human body characteristics.
  • Cardiac Output Describes the amount of blood pumped by the heart per unit time.
  • SVR Systemic Vascular Resistance
  • the description can be combined with specific formulas. For example, based on the linear fitting relationship between ejection time and heart rate, the ejection time corresponding to the heart rate estimate is determined.
  • the human body surface area corresponding to the target object can be calculated based on the gender data, height, and weight included in the profile data.
  • the human body surface area BSA is calculated based on the target object's health file information (Height, Weight) and Xu Shengwen's formula.
  • the stroke volume SV corresponding to the target object can be determined based on the age included in the body profile data. It can be determined according to the following formula (1):
  • the estimated pulse pressure value (Pulse pressure, Pp) corresponding to the target object can be calculated, which can be determined by the following formula (2):
  • CE 0.013*Weight-0.007*Age-0.004*HR+1.307
  • Pm CO*SVR
  • CO cardiac output
  • SVR systemic vascular resistance
  • the blood pressure measurement results and the collection time corresponding to the multi-frame images can be stored in the blood pressure measurement record of the target object.
  • the blood pressure measurement record can store the blood pressure measurement conditions of the target object in each period. , and can also generate corresponding blood pressure measurement reports based on blood pressure measurement records within a certain preset period, thereby facilitating real-time monitoring of target objects.
  • real-time blood pressure monitoring can reduce the risk of excessive blood pressure. Or the phenomenon of dangerous driving behavior caused by too low, improve driving safety.
  • the blood pressure measurement report can be, for example, smoothing multiple measurement results within 5 minutes, or generating a dynamic change report of blood pressure over a period of time (such as blood pressure comparison and change trends at multiple different periods within a day). In addition, it can also be There are no specific restrictions on reports related to blood pressure monitoring in other dimensions.
  • blood pressure measurement results can be stored in association with the target subject's body profile data.
  • body profile data such as age and weight change
  • the blood pressure measurement results will also change accordingly, which facilitates the analysis of the target object's physical condition over a longer period of time, and can also detect abnormal body indicators in the event of abnormal body indicators. Provide timely reminders.
  • embodiments of the present disclosure can also display the blood pressure measurement results to provide better cabin services to the target object through the displayed blood pressure measurement results.
  • the time domain signal in order to improve the accuracy of physiological state detection, can be converted into the frequency domain, and more useful information can be analyzed based on the converted frequency domain signal. For example, each frequency component can be determined. The amplitude distribution and energy distribution are obtained, thereby obtaining the frequency values of the key amplitude and energy distribution.
  • the physiological state value of the target object can be determined based on the peak value of the frequency domain signal.
  • the peak value pmax of the frequency domain signal can be determined.
  • the original heart rate measurement value can be obtained by summing pmax and the heart rate reference value.
  • pmax represents the heart rate change
  • the heart rate reference value can be determined by The lower limit of the heart rate estimation range can be determined, and the influence of factors such as video frame rate and frequency domain signal length can also be considered to adjust the heart rate reference value.
  • blood oxygen saturation you can use red light (600 to 800nm) and near-red light area (800 to 1000nm) to detect the time domain signals of HbO2 and Hb respectively, and then calculate the corresponding ratio to get the blood oxygen saturation; for For heart rate variability, after extracting the time domain signal, several intervals are obtained by calculating the distance between each two adjacent wave peaks and combining it with the frame rate, and taking the standard deviation of these intervals (Standard Deviation of NN Intervals, SDNN), that is Get heart rate variability.
  • SDNN Standard Deviation of NN Intervals
  • Respiratory frequency detection is similar to heart rate detection. The difference is that the range of respiratory frequency is different from the range of heart rate, and the corresponding reference value settings are different. Respiratory frequency detection can be implemented based on the same method as above.
  • the embodiments of the present disclosure implement is the detection of physiological status in multi-frame images, that is, the image change information corresponding to the multi-frame images can represent the changes in physiological status.
  • the physiological state detection results determined regarding the video stream may be updated as the collection of image frames continues.
  • face detection can be performed on the images in the new video stream to extract the target object in the cabin. face image, and then determine at least one smooth area in each frame of the face image, and weight the at least one smooth area or filter the area based on the area attribute information to obtain the information to be extracted related to the physiological state, and based on the area to be extracted Extract information to update the physiological state detection results. If the preset detection duration is not reached, update again based on the acquired new video stream until the preset detection duration is reached, and the updated physiological status detection results are obtained.
  • face detection can be performed on the images in the new video stream to extract the facial image of the target object in the cabin, and then determine At least one smooth area in the facial image, and determine at least one area of interest whose image brightness meets the preset conditions from the at least one smooth area, so that the physiological state detection results can be performed based on the image information of the at least one area of interest.
  • Update if the preset detection duration is not reached, update again based on the acquired new video stream until the preset detection duration is reached, and the updated physiological state detection result is obtained.
  • face detection can be performed on the images in the new video stream to extract the face of the target object in the cabin. image, and then determine at least one smooth area in the facial image, and for each smooth area, generate a time domain brightness signal corresponding to the smooth area based on the pixel brightness information of at least one color channel of the smooth area in the multi-frame facial image, and then Based on the area of each smooth area, perform weighting processing on the time domain brightness signal corresponding to at least one smooth area to obtain an area weighted time domain brightness signal, and update the physiological state detection results based on the area weighted time domain brightness signal. , if the preset detection duration is not reached, it is updated again based on the acquired new video stream until the preset detection duration is reached, and the updated physiological state detection result is obtained.
  • face detection can be performed on the images in the new video stream to extract the face of the target object in the cabin. image, and then determine at least one smooth area in the facial image, and in the case of determining the first contribution degree and the second contribution degree of each smooth area to the physiological state detection result, based on the first contribution degree and the second contribution degree, The image information of at least one smooth area is fused to obtain the fused image information, and the physiological state detection result is updated based on the fused image information. If the preset detection time is not reached, it is updated again based on the new video stream obtained. Until the preset detection duration is reached, the updated physiological status detection results are obtained.
  • the video stream can be continuously obtained within 30s.
  • the heart rate measurement is calculated based on multiple frames of the starting video stream (e.g. within the first 5 seconds of the video stream), this is still within 30 seconds.
  • the number of image frames increases.
  • a new heart rate measurement value can be calculated for each additional frame or for each additional n frames, and then smoothed through the sliding average. The measurement ends after 30 seconds, and the final value is obtained. Measurement results.
  • a generated video stream to remind the target can be generated based on the duration of the acquired video stream and the preset detection duration.
  • Detection process reminder signal for the required detection duration of the object. For example, if the duration of the acquired video stream (that is, the detection time for which the current target object's physiological state detection has been continued) reaches 25 seconds, and the default detection duration is 30 seconds, it can be sent A voice or screen prompt about "Please stay still, there are still 5 seconds to complete the test"; or, when the current target object's physiological state detection reaches 30 seconds, a voice or screen prompt about "Measurement Completed" will be issued. .
  • embodiments of the present disclosure can also display physiological state detection results to provide better cabin services for target objects through the displayed physiological state detection.
  • the physiological status detection results of the target object can be transmitted to the display screen in the cabin for display on the display screen.
  • the cabin personnel can monitor their own physiological status in real time and also monitor their own physiological status. If the physiological state of the target object is abnormal, seek medical treatment or take other necessary measures in a timely manner; at the same time, the physiological state detection results of the target object can also be transmitted to the server of the physiological state detection application, so that the target object can obtain the detection results after requesting the physiological state detection application through the physiological state detection application. If the result is obtained, the physiological state detection result is sent to the terminal device used by the target object through the server.
  • the physiological status detection results of the target object can be recorded on the server side, and the physical status detection results can also be statistically analyzed on the server side. For example, the physiological status statistical results of the historical month and week can be determined. In this way, on When the target object initiates a physiological state detection application request, the physiological state detection results, statistical results, etc. can be sent to the target object's terminal device to achieve a more comprehensive physiological state assessment.
  • the above-mentioned physiological state detection application can be a specific application (Application, APP) used for physiological state detection.
  • the APP can be used to respond to the acquisition request of the detection results related to the target object, and then realize the presentation of the results on the APP, and more Practical.
  • the writing order of each step does not mean a strict execution order and does not constitute any limitation on the implementation process.
  • the specific execution order of each step should be based on its function and possible The internal logic is determined.
  • the embodiments of the present disclosure also provide a physiological state detection device corresponding to the physiological state detection method. Since the principle of solving the problem of the device in the embodiments of the present disclosure is similar to the above-mentioned physiological state detection method of the embodiments of the present disclosure, therefore The implementation of the device may refer to the implementation of the method.
  • the device includes: an acquisition part 301, configured to acquire a video stream collected by a camera device; an extraction part 302, configured to obtain the video stream from the video stream. Extract multiple frames of facial images of the target object from the multi-frame images in the facial image; the determining part 303 is configured to determine at least one smooth region in the facial image; the processing part 304 is configured to determine at least one smooth region based on the region attribute information.
  • the area is weighted or filtered to obtain the information to be extracted related to the physiological state; the detection part 305 is configured to extract physiological state information based on the information to be extracted to obtain the physiological state detection result of the target object.
  • the determining part 303 is also configured to extract facial feature points from the facial image to obtain the positions of multiple preset key feature points in the facial image; based on the multiple preset key The positions of the feature points determine at least one smooth area in the facial image corresponding to the plurality of preset key feature points.
  • the area attribute information includes area brightness
  • the information to be extracted related to the physiological state includes image information of at least one area of interest.
  • the processing part 304 is also configured to determine from at least one smooth area.
  • the region brightness meets at least one region of interest that meets the preset conditions;
  • the detection part 305 is also configured to extract physiological state information from the image information of at least one region of interest to obtain a physiological state detection result.
  • the processing part 304 is further configured to determine the image brightness threshold corresponding to the face image and the regional image brightness of each smooth region; according to the image brightness threshold and the regional image brightness, at least one smooth region is Filtering is performed to obtain at least one candidate smooth area; among the at least one candidate smooth area, at least one area of interest whose brightness meets a preset brightness condition is selected.
  • the image information of the region of interest includes pixel brightness information of three different color channels
  • the detection part 305 is further configured to determine the representation based on the pixel brightness information of the three color channels of the region of interest.
  • the time domain signal of the physiological state of the target object perform frequency domain conversion on the time domain signal to obtain a frequency domain signal representing the physiological state of the target object; determine the physiological state detection result of the target object based on the peak value of the frequency domain signal.
  • the detection part 305 is further configured to determine, based on the pixel brightness information of the three color channels corresponding to the area of interest, the time domain brightness signal of each color channel corresponding to the area of interest;
  • the area of interest corresponds to the time domain brightness signal of three different color channels and is subjected to principal component analysis to obtain the time domain signal.
  • the area attribute information includes the area of the area, and the information to be extracted related to the physiological state includes a weighted time-domain brightness signal.
  • the processing part 304 is also configured to perform a calculation for each smooth area based on multiple frames.
  • the pixel brightness information of at least one color channel of the smooth area in the facial image generates a time domain brightness signal corresponding to the smooth area; based on the area of each smooth area, the time domain brightness signal corresponding to at least one smooth area is weighted to obtain Area-weighted time-domain brightness signal;
  • the detection part 305 is also configured to extract physiological state information based on the area-weighted time-domain brightness signal to obtain a physiological state detection result.
  • the smooth area corresponds to pixel brightness information of three different color channels
  • the processing part 304 is further configured to based on the pixel brightness information of the smooth area corresponding to three color channels in the multi-frame facial image. , determine the smooth area corresponding to the time domain brightness signal of each color channel; for each color channel, based on the area of each smooth area, weight the time domain brightness signal of at least one smooth area under the color channel to obtain The area-weighted time-domain brightness signal under the color channel; the detection part 305 is also configured to perform principal component analysis on the area-weighted time-domain brightness signal of the smooth area under multiple different color channels to obtain the representation of the target object.
  • the time domain signal representing the physiological state of the target object is subjected to frequency domain processing, and the physiological state detection result is determined based on the peak value of the frequency domain signal obtained by the frequency domain processing.
  • the regional attribute information includes regional area and regional brightness
  • the information to be extracted related to the physiological state includes fused image information.
  • the processing part 304 is also configured to determine each smoothed region according to the area of each smoothed region. The first contribution degree of the area to the physiological state detection result; according to the pixel brightness information of each smooth area, the second contribution degree of each smooth area to the physiological state detection result is determined; based on the first contribution degree and the second contribution degree, at least one
  • the image information in the smooth area is fused to obtain fused image information; the detection part 305 is also configured to extract physiological state information from the fused image information to obtain physiological state detection results.
  • the first contribution includes an area weight
  • the second contribution includes a brightness weight
  • the processing part 304 is further configured to determine each smooth area based on the product of the area weight and the brightness weight respectively corresponding to each smooth area.
  • the total weight value corresponding to the smooth area respectively; perform weighted fusion of the image information of at least one smooth area based on the corresponding total weight value to determine the fused image information.
  • the fused image information includes a fused time domain brightness signal
  • the processing part 304 is also configured to perform time domain processing on the brightness value of the smooth area in the multi-frame image for each smooth area, Determine the time domain brightness signal corresponding to the smooth area; based on the first contribution and the second contribution, fuse the time domain brightness signals corresponding to the multiple smooth areas to obtain a fused time domain brightness signal;
  • the detection part 305 is configured as The fused time domain brightness signal is subjected to frequency domain processing, and the physiological state result is determined based on the peak value of the frequency domain signal obtained by the frequency domain processing.
  • the processing part 304 is also configured to search for body profile data matching the target object from the preset archive based on the facial image; the detection part 305 is also configured to search for body profile data matching the target object based on the multi-frame face image.
  • the image information of the body image is processed in time domain and frequency domain to obtain the heart rate estimate of the target object; based on the body profile data and heart rate estimate, the blood pressure value of the target object is estimated to obtain the blood pressure measurement result.
  • the processing part 304 is further configured to search for body profile data corresponding to the facial features of the target object from the preset archive based on the facial features of the target object extracted from the facial image. ;Based on the identification of the target object recognized by the facial image, search for the body profile data corresponding to the identification from the preset archive library.
  • the detection part 305 is further configured to determine the ejection time corresponding to the heart rate estimate based on the linear fitting relationship between the ejection time and the heart rate; and, based on the gender data included in the body profile data , height and weight, calculate the human body surface area corresponding to the target object; and, based on the weight and age included in the body file data, and the heart rate estimate of the target object, determine the heart elasticity value of the target object; based on the ejection time, human body surface area, body file
  • the age and heart rate estimate included in the data are used to determine the stroke volume corresponding to the target object; based on the ratio between the stroke volume corresponding to the target object and the heart elasticity value, the pulse pressure estimate corresponding to the target object is calculated; based on the target
  • the target's corresponding pulse pressure estimate and the gender data included in the body profile data are used to calculate the systolic blood pressure and diastolic blood pressure values of the target subject.
  • the extraction part 302 is further configured to determine the facial image of the target object from the detected facial image according to the face detection result of the video stream and the specified riding location; wherein , the specified ride position is used to indicate the position of the target object being measured.
  • the device further includes: a display part configured to transmit the physiological state detection result of the target object to a display screen in the vehicle cabin for display on the display screen; a sending part configured to The physiological state detection result of the target object is transmitted to the server of the physiological state measurement application, so that when the target object requests to obtain the detection result through the physiological state application, the physiological state detection result is sent to the terminal device used by the target object through the server.
  • the detection part 304 is also configured to, when a new video stream is obtained, repeatedly perform the following steps until the preset detection duration is reached, and obtain the updated physiological state detection result: from new Extract multi-frame facial images of the target object from the video stream; determine at least one smooth area in each frame of facial image; and based on the area attribute information, perform weighting processing or area filtering on at least one smooth area to obtain information related to the physiological state. information to be extracted, and the physiological state detection results are updated based on the image information to be extracted.
  • the device includes: a reminder part configured to generate a detection process reminder signal according to the duration of the acquired video stream and the preset detection duration, and the detection process reminder signal is used to remind the target object of the required detection. duration.
  • FIG. 4 is a schematic structural diagram of the electronic device provided by an embodiment of the present disclosure, it includes: a processor 401, a memory 402, and a bus 403.
  • the memory 402 stores machine-readable instructions executable by the processor 401 (for example, the execution instructions corresponding to the acquisition part 301, the extraction part 302, the determination part 303, the processing part 304 and the detection part 305 in the device in Figure 3, etc.).
  • the processor 401 and the memory 402 communicate through the bus 403.
  • the machine-readable instructions are executed by the processor 401, the following processing is performed:
  • Physiological state information is extracted based on the information to be extracted, and the physiological state detection results of the target object are obtained.
  • Embodiments of the present disclosure also provide a computer-readable storage medium.
  • a computer program is stored on the computer-readable storage medium. When the computer program is run by a processor, it executes the physiological state detection method described in the above method embodiment.
  • the storage medium may be a volatile or non-volatile computer-readable storage medium.
  • Embodiments of the present disclosure also provide a computer program product.
  • the computer program product carries program code.
  • the instructions included in the program code can be used to execute the physiological state detection method described in the above method embodiment.
  • Embodiments of the present disclosure also provide a computer program, which computer program includes computer readable code.
  • the processor of the electronic device executes to implement the above method.
  • the above-mentioned computer program product can be specifically implemented by hardware, software or a combination thereof.
  • the computer program product is embodied as a computer storage medium.
  • the computer program product is embodied as a software product, such as a Software Development Kit (SDK), etc. wait.
  • SDK Software Development Kit
  • a unit described as a separate component may or may not be physically separate.
  • a component shown as a unit may or may not be a physical unit, that is, it may be located in one place, or it may be distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.
  • each functional unit in various embodiments of the present disclosure may be integrated into one processing unit, or each unit may exist physically alone, or two or more units may be integrated into one unit.
  • the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium that is executable by a processor.
  • the technical solution of the present disclosure is essentially or the part that contributes to the existing technology or the part of the technical solution can be embodied in the form of a software product.
  • the computer software product is stored in a storage medium, including Several instructions are used to cause an electronic device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present disclosure.
  • the aforementioned storage media include: U disk, mobile hard disk, read-only memory (ROM), random access memory (Random Access Memory, RAM), magnetic disk or optical disk and other media that can store program code. .
  • Embodiments of the present disclosure provide a physiological state detection method, device, electronic device, storage medium and program, wherein the method includes: obtaining a video stream collected by a camera device; extracting the target object from the multi-frame images in the video stream. Multi-frame facial images; determine at least one smooth area in the facial image; perform weighting processing or area screening on the at least one smooth area based on the area attribute information to obtain information to be extracted related to the physiological state; The information to be extracted is used to extract physiological state information, and the physiological state detection results of the target object are obtained.

Landscapes

  • Engineering & Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Theoretical Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • Image Processing (AREA)

Abstract

本公开实施例提供了一种生理状态检测方法、装置、电子设备、存储介质及程序，其中，该方法包括：获取摄像设备采集的视频流；从视频流中的多帧图像中提取目标对象的多帧脸部图像；确定所述脸部图像中的至少一个平滑区域；基于区域属性信息，对所述至少一个平滑区域进行加权处理或区域筛选，得到与生理状态有关的待提取信息；对所述待提取信息进行生理状态信息提取，得到目标对象的生理状态检测结果。本公开实施例基于图像处理方式实现生理状态检测，可随时随地进行实时地测量，实用性更佳，且能够通过预设条件的约束提升测量的准确度。

Description

生理状态检测方法、装置、电子设备、存储介质及程序
相关申请的交叉引用
本公开实施例基于申请人为上海商汤临港智能科技有限公司，申请日为2022年03月31日提交四个中国专利，分别为：中国专利申请号为202210344726.1、申请名称为“一种生理状态检测方法、装置、电子设备及存储介质”，中国专利申请号为202210344718.7、申请名称为“一种生理状态检测方法、装置、电子设备及存储介质”，中国专利申请号为202210346427.1、申请名称为“一种血压测量方法、装置、电子设备及存储介质”，中国专利申请号为202210346417.8、申请名称为“一种生理状态检测方法、装置、电子设备及存储介质”的中国专利申请提出，并要求以上四个中国专利申请的优先权，该中国专利申请的全部内容以引用的方式并入本公开中。
技术领域
本公开涉及计算机技术领域，涉及一种生理状态检测方法、装置、电子设备、存储介质及程序。
背景技术
准确的生理状态数据是分析人体变异性的基础，所以对生理状态的检测有着重要的意义。以安全驾驶场景为例，有效的生理状态检测可以帮助了解车内乘员的生理状态，从而为安全驾驶提供辅助性的决策。相关技术中，主要依赖专用的检测设备，如血压仪、心率仪、血氧仪等设备进行生理状态检测，除此之外，还可以借助集成有相关感应元器件的智能手表、智能手环等穿戴设备实现生理状态的测量。
可知，上述检测方案需要借助专用的仪器进行接触式测量，这为检测带来了不便，从而不能很好地满足诸如安全驾驶场景的需要。
发明内容
本公开实施例至少提供一种生理状态检测方法、装置、电子设备、存储介质及程序。
本公开实施例提供了一种生理状态检测方法，包括：获取摄像设备采集的视频流；从所述视频流中的多帧图像中提取目标对象的多帧脸部图像；确定所述脸部图像中的至少一个平滑区域；基于区域属性信息，对所述至少一个平滑区域进行加权处理或区域筛选，得到与生理状态有关的待提取信息；对所述待提取信息进行生理状态信息提取，得到所述目标对象的生理状态检测结果。
本公开实施例还提供了一种生理状态检测装置，包括：获取部分，被配置为获取摄像设备采集的视频流；提取部分，被配置为从所述视频流中的多帧图像中提取目标对象的多帧脸部图像；确定部分，被配置为确定所述脸部图像中的至少一个平滑区域；处理部分，被配置为基于区域属性信息，对所述至少一个平滑区域进行加权处理或区域筛选，得到与生理状态有关的待提取信息；检测部分，被配置为对所述待提取信息进行生理状态信息提取，得到所述目标对象的生理状态检测结果。
本公开实施例还提供了一种电子设备，包括：处理器、存储器和总线，所述存储器存储有所述处理器可执行的机器可读指令，当电子设备运行时，所述处理器与所述存储器之间通过总线通信，所述机器可读指令被所述处理器执行时执行如上述所述的生理状态检测方法。
本公开实施例还提供了一种计算机可读存储介质，该计算机可读存储介质上存储有 计算机程序，该计算机程序被处理器运行时执行如上述所述的生理状态检测方法的。
本公开实施例还提供了一种计算机程序，所述计算机程序包括计算机可读代码，在所述计算机可读代码在电子设备中运行的情况下，所述电子设备的处理器执行用于实现所述的生理状态检测方法。
本公开实施例提供的生理状态检测方法、装置、电子设备、存储介质及程序，在获取到视频流的情况下，可以从视频流中提取目标对象的多帧脸部图像，并可以确定每帧脸部图像中的至少一个平滑区域，而后可以基于平滑区域的区域属性信息，对确定的至少一个平滑区域进行加权或筛选，从而可基于进行加权或筛选的结果进行处理得到所述目标对象的生理状态检测结果。相比相关技术中需要借助专用的仪器进行接触式测量所带来的测量不便的问题，本公开基于图像处理方式实现生理状态检测，可随时随地进行实时地测量，实用性更佳，如此，能够提升测量的准确度和灵活性。
为使本公开的上述目的、特征和优点能更明显易懂，下文特举较佳实施例，并配合所附附图，作详细说明如下。
附图说明
为了更清楚地说明本公开实施例的技术方案，下面将对本公开实施例中所需要使用的附图进行说明。此处的附图被并入说明书中并构成本说明书的一部分，这些附图示出了符合本公开的实施例，并与说明书一起用于说明本公开的技术方案。
图1示出了本公开实施例所提供的第一种生理状态检测方法的流程示意图；
图2A示出了针对摄像头拍摄的脸部图像可以提取出的脸部特征点的示意图；
图2B示出了本公开实施例所提供的第二种生理状态检测方法的流程示意图；
图2C示出了本公开实施例所提供的第三种生理状态检测方法的流程示意图；
图2D示出了本公开实施例所提供的第四种生理状态检测方法的流程示意图；
图2E示出了本公开实施例所提供的第五种生理状态检测方法的流程示意图；
图3示出了本公开实施例所提供的一种生理状态检测装置的示意图；
图4示出了本公开实施例所提供的一种电子设备的示意图。
具体实施方式
为使本公开实施例的目的、技术方案和优点更加清楚，下面将结合本公开实施例中附图，对本公开实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例仅仅是本公开一部分实施例，而不是全部的实施例。通常在此处附图中描述和示出的本公开实施例的组件可以以各种不同的配置来布置和设计。因此，以下对在附图中提供的本公开的实施例的详细描述并非旨在限制要求保护的本公开的范围，而是仅仅表示本公开的选定实施例。基于本公开的实施例，本领域技术人员在没有做出创造性劳动的前提下所获得的所有其他实施例，都属于本公开保护的范围。
应注意到：相似的标号和字母在下面的附图中表示类似项，因此，一旦某一项在一个附图中被定义，则在随后的附图中不需要对其进行进一步定义和解释。
本文中术语“和/或”，仅仅是描述一种关联关系，表示可以存在三种关系，例如，A和/或B，可以表示：单独存在A，同时存在A和B，单独存在B这三种情况。另外，本文中术语“至少一种”表示多种中的任意一种或多种中的至少两种的任意组合，例如，包括A、B、C中的至少一种，可以表示包括从A、B和C构成的集合中选择的任意一个或多个元素。
经研究发现，相关技术中，主要依赖专用的检测设备，如血压仪、心率仪、血氧仪等设备进行生理状态检测，除此之外，还可以借助集成有相关感应元器件的智能手表、智能手环等穿戴设备实现生理状态的测量。可知，上述检测方案需要借助专用的仪器进 行接触式测量，这为检测带来了不便，从而不能很好地满足诸如安全驾驶场景的需要。
为了解决上述问题，相关技术中提供了一种无接触式检测生理状态的方案，即远程光电容积脉搏波描记法(remote Photoplethysmographic，rPPG)，该方法可以借助目前广泛使用的带摄像头的手机终端就可以完成检测，无需额外的硬件成本，使用上非常方便。而rPPG方法目前的瓶颈在于检测精度逊色于一些专用的检测设备，同时也容易受到外界光线的影响。另外，利用rPPG方法的生理特征检测，需要让被检测对象保持一段时间的静止，只能用于主动检测。
传统基于rPPG的生理特征监测需要选取一个感兴趣区域(Region of Interest，ROI)，在实际相机成像中可能出现画面过暗、过亮的情况，单个ROI区域往往是固定一个人脸范围作为ROI区域，容易选取到曝光不理想的区域进行信号提取；或者，在实际相机成像中可能由于人脸与摄像头光轴存在夹角，ROI区域成像后在图像中所占的像素面积可能过小，导致ROI区域信息量少，从而难以获得理想的检测结果。
基于上述研究，本公开实施例提供了一种通过预设条件的约束，例如：基于图像亮度选取最优ROI区域，或基于区域面积进行加权，或基于区域面积和区域亮度进行加权，实现生理状态检测的方案，以得到更接近实际情况的信号量，提升检测精度。
为便于对本实施例进行理解，首先对本公开实施例所公开的一种生理状态检测方法进行详细介绍，本公开实施例所提供的生理状态检测方法的执行主体一般为具有一定计算能力的电子设备，该电子设备例如包括：终端设备或服务器或其它处理设备，终端设备可以为用户设备(User Equipment，UE)、移动设备、用户终端、蜂窝电话、无绳电话、个人数字助理(Personal Digital Assistant，PDA)、手持设备、车载设备、可穿戴设备等。在本公开的一些实施例中，该生理状态检测方法可以通过处理器调用存储器中存储的计算机可读指令的方式来实现。
参见图1所示，为本公开实施例提供的生理状态检测方法的流程图，该方法包括步骤S101至S105，其中：
S101：获取摄像设备采集的视频流；
S102：从视频流中的多帧图像中提取目标对象的多帧脸部图像；
S103：确定脸部图像中的至少一个平滑区域；
S104：基于区域属性信息，对至少一个平滑区域进行加权处理或区域筛选，得到与生理状态有关的待提取信息；
S105：对待提取信息进行生理状态信息提取，得到目标对象的生理状态检测结果。
本公开实施例中的生理状态检测方法可以应用于需要进行生理状态检测的汽车领域，本公开实施例可以实现针对车舱环境内人体的生理状态检测。同时基于本公开实施例提供的生理状态检测方法，在得到车舱环境内有关目标对象的生理状态检测结果的情况下，可以及时地了解目标对象是否存在异常的身体状况，并能够在身体状态出现异常的情况下，及时地进行提醒或者提供援助，如此，能够为安全驾驶提供更为充足的现实可能性。
除此之外，本公开实施例还可以应用有诸如医疗、家居生活等其它任何需要进行生理状态检测的相关领域。考虑到汽车驾驶领域的广泛应用，接下来多以汽车领域进行示例说明。同时本公开实施例中的视频流可以是通过摄像设备(如汽车场景中的车内固定摄像头)采集得到的，还可以是用户终端自带摄像头采集得到的，还可以是其它方式采集得到的。
为了能实现针对特定目标对象的生理状态检测，有关摄像头的安装位置可以是基于特定的目标对象预先设置的，例如，为了实现针对车辆内的驾驶员的生理状态检测，这里的摄像头可以安装于拍摄范围覆盖驾驶区域的位置，例如车辆的A柱内侧、控制台上、或者方向盘位置；再如，为了实现针对车辆内的包括驾驶员和乘客在内的多种乘车属性 的乘员的生理状态检测，这里的摄像头可以安装于车内后视镜、顶部装饰件、阅读灯等拍摄范围可覆盖车舱内多个座位区域的位置。
在汽车领域的实际应用中，还可以采用驾驶员监控系统(Driver Monitoring System，DMS)包含的车内图像采集装置实现有关驾驶区域的视频流的采集，或者，可以采用乘员监控系统(Occupant Monitoring System，OMS)包含的车内图像采集装置实现有关乘车区域的视频流的采集。
考虑到面部血管流过产生的皮肤色彩及亮度变化，可以反映心跳、呼吸等生理状态，这里可以首先对视频流中的多帧图像进行脸部检测，提取出车舱内的目标对象的多帧脸部图像，继而针对脸部图像实现生理状态信息的提取。
目标对象是人体生理状态检测的测量对象，其可以是停留在指定位置的人员对象，或者具有特定身份属性的人员对象。例如，在汽车领域中，目标对象可以是特定乘车属性的对象，例如驾驶员、副驾座位的乘车人；或者，目标对象可以是身份标识为预先使用面部信息注册的对象，例如通过应用程序注册的车主；或者，目标对象也可以是车内的任意一个乘员，可通过对车舱内的视频流进行面部检测定位出至少一个乘员，将检测出的一个或多个乘员作为目标对象。
在进行脸部检测的过程中，在一帧图像上可能会出现多个对象的面部的情况。一些场景中，可以选择针对某一乘车位置的乘员进行生理状态检测，即将该乘车位置的乘员作为目标对象。为了实现针对车舱内的目标对象的生理状态检测，这里，可以根据多帧图像的脸部检测结果和指定的乘车位置，从检测出的脸部图像中确定出目标对象的多帧脸部图像，其中，指定的乘车位置用于指示被测量的目标对象的位置。
车舱内用于采集视频流的摄像头在车辆内部空间的相对位置是固定的，可以根据摄像头的位置，将其采集的图像按照座位区域划分，例如对于5座私家车可以划分为：驾驶座对应的图像区域、副驾驶座对应的图像区域、后排左侧座位对应的图像区域、后排右侧座位对应的图像区域、后排中间座位对应的图像区域。根据车内各乘员对象的面部在图像中的位置以及各图像区域的坐标范围，可以确定出各乘员对象的面部落入的图像区域，进而确定出在指定的乘车位置的乘员对象为目标对象。
在实际应用中，OMS一般会拍摄整个车舱空间内的图像，可能拍到多个人，可以手动选择“前车乘车位”、“后座乘车位”，来指定车舱空间内要测量的目标对象，这时本公开实施例可以对图像中相应区域的人脸进行测量。DMS针对主驾驶区域进行拍摄，在其拍摄到的对象仅包含司机一人的情况下，可以无需指定对象。
在本公开的一些实施例中，生理状态，例如心率、呼吸频率、血氧、血压等往往需要一定时长的监测才可以进行评估，因而，本公开实施例中采用持续一段时间的视频流内的多帧脸部图像所对应的图像变化信息实现生理状态信息的提取，这样所提取出的生理状态检测结果也更为符合实际场景的需要。
在本公开的一些实施例中，在基于脸部图像进行图像变化信息分析的过程中，由于受到诸如人脸转动、遮挡或者外界光线所造成的ROI区域(即对应本公开实施例提供的生理状态检测方法中的平滑区域)丢失而存在无法继续检测的问题，本公开实施例可以是利用脸部图像中的多个平滑区域进行检测；这样，即使其中一个平滑区域丢失，在其它平滑区域表现良好的情况下，也能够继续进行检测，使得使用场景更加广泛。
在一些实施例中，在考虑到由于人脸与摄像头光轴存在夹角，不同平滑区域在摄像头成像所占的像素面积是存在差异的，因而本公开实施例还可以通过选取的多个平滑区域并通过平滑区域在画面中的像素面积进行加权，得到更接近实际情况的信号量，进而能够提升检测精度。
在一些实施例中，还考虑到由于人脸与摄像头光轴存在夹角，不同平滑区域在摄像头成像所占的像素面积以及所对应的像素亮度是存在差异的，因而这里可以通过平滑区 域在画面中的像素面积以及对应的像素亮度进行加权，得到更接近实际情况的信号量，进而能够提升检测精度。
其中，针对每帧脸部图像均可以确定至少一个平滑区域，该平滑区域可以是对应脸部图像中的一个或多个脸部平滑子区域进行筛选确定的，其中，脸部平滑子区域可以是平滑的连通区域，这里，可以要求该连通区域为可通过脸部关键点定位出的矩形、圆形、或椭圆形等指定形状，每个连通区域内不包含脸部的眼睛、鼻子、嘴巴、眉毛等非平滑特征。该连通区域一定程度上具有更均匀的反射率，从而可以捕捉到更为有效的面部血管流过产生的皮肤色彩及亮度变化，继而可以实现更为精准的生理状态检测。
在确定每帧脸部图像的多个脸部平滑子区域的情况下，本公开实施例提供的生理状态检测方法可以首先基于有关图像亮度设置的预设条件进行至少一个平滑区域的确定，继而基于至少一个平滑区域的图像信息所对应的图像变化信息进行生理状态信息的提取，这里所提取的生理状态检测结果可以是包括心率、呼吸频率、血氧、血压等中的至少一个在内的检测结果。
本公开实施例中，可以通过对多个脸部平滑子区域的图像亮度选择最优的ROI区域作为平滑区域，从而可以针对该平滑区域进行光电容积描记(Photoplethysmography，PPG)信号提取，从而提升rPPG方法的检测精度以及检测中抗干扰的能力。
其中，有关图像亮度所设置的预设条件可以是滤除明显过曝或者明显过暗的相关脸部平滑子区域的条件，这是考虑到明显过曝的脸部平滑子区域对应的图像亮度异常的高，在这种情况下所进行的PPG信号提取不够准确，进而导致最终得到的生理状态检测结果的检测精度较低，同理，对于明显过暗的脸部平滑子区域而言，其图像亮度异常的低，同样导致较低的检测精度。
在本公开的一些实施例中，首先可以对脸部图像进行脸部特征点提取，继而可以基于提取出的脸部特征点，确定脸部图像中的至少一个平滑区域。即上述步骤S103可以通过以下过程来实现：
步骤一，对脸部图像进行脸部特征点提取，得到脸部图像中的多个预设关键特征点的位置；
步骤二，基于多个预设关键特征点的位置，确定与多个预设关键特征点对应的脸部图像中的至少一个平滑区域。
其中，上述提取脸部特征点的过程可以是利用人脸关键点检测算法实现的，例如，可以是预先设置有关标准人脸图像的脸部特征点，这里的标准人脸可以是包括五官在内的正对摄像头拍摄的人脸图像；这样，在对每帧图像提取出的目标对象的脸部图像进行脸部特征点提取的过程中，可以基于提取的目标对象的脸部图像与标准人脸图像之间的比对情况来确定各个脸部特征点。例如，可以是眉毛特征点、鼻梁特征点、鼻尖特征点、脸颊特征点、嘴角特征点等具有明显脸部特性的相关特征点。
本公开实施例中，基于脸部图像中的多个预设关键特征点的位置可以确定脸部图像中的一个或多个平滑区域。这里，平滑区域可以是矩形区域，还可以是具有连通性形状的其它区域，本公开实施例对此不做限制，接下来多以矩形区域为例进行说明。
在实际应用中，这里的平滑区域可以是基于眉毛特征点确定的额头平滑区域，基于脸颊特征点、鼻梁特征点和鼻尖特征点确定的左侧上部脸颊平滑区域和右侧上部脸颊平滑区域，以及基于脸颊特征点、鼻尖特征点和嘴角特征点确定的左侧下部脸颊平滑区域和右侧下部脸颊平滑区域。
在本公开的一些实施例中，在未出现区域遮挡的情况下，上述五个区域(额头平滑区域、左侧上部脸颊平滑区域、右侧上部脸颊平滑区域、左侧下部脸颊平滑区域以及右侧下部脸颊平滑区域)可以在一帧脸部图像上同时被提取到。
同时，在出现区域遮挡的情况下，一帧脸部图像实际所能够提取出的区域可以依照 实际情况来确定。本公开实施例中可以按照如下步骤来确定被遮挡后的平滑区域：
步骤一、在提取出的脸部特征点中缺失预设关键特征点的情况下，确定缺失的预设关键特征点对应的脸部子区域为缺失区域，并确定预定义的多个脸部子区域中除缺失区域之外的其他脸部子区域为非缺失区域；
步骤二、根据提取出的脸部特征点中非缺失区域对应的预设关键特征点的位置确定非缺失区域的边界，以确定至少一个平滑区域。
在本公开的一些实施例中，提取出的脸部特征点中缺失预设关键特征点，说明对应的脸部子区域被遮挡，可以确定其它未缺失的脸部关键点对应的脸部子区域为非缺失区域。在基于非缺失区域对应的预设关键特征点的位置确定非缺失区域的边界的情况下，即确定出平滑区域。可知的是，即使发生了遮挡，本公开实施例也可以按照上述方法较为精准地提取出平滑区域。
如图2A所示，为针对摄像头拍摄的脸部图像可以提取出的脸部特征点的示意图；其中，该脸部提取的共计106特征点。这里，基于脸部特征点的坐标信息，可以筛选5个平滑区域，可参见图2A，其中，区域1可以是通过两侧眉毛的两个特征点构造的区域1的矩形ROI；区域2是左侧的一个脸颊区域，可以通过脸部左侧边缘特征点、鼻梁特征点、左眼特征点的位置构造区域2的矩形ROI；区域3是右侧的一个脸颊区域，可以通过脸部右侧边缘特征点、鼻梁特征点、右眼特征点的位置构造区域3的矩形ROI，区域4是左侧的另一个脸颊区域，可以通过脸部左侧边缘特征点、左侧鼻翼特征点、左嘴角特征点的位置构造区域4的矩形ROI，区域5是右侧的另一个脸颊区域，可以通过脸部右侧边缘特征点、右侧鼻翼特征点、右嘴角特征点的位置构造区域5的矩形ROI。
本公开实施例在进行脸部特征点提取的过程中，还可以结合脸部姿态来实现。这里，首先可以根据脸部图像检测得到的目标对象的脸部姿态，而后根据脸部姿态对脸部图像进行脸部特征点提取。
其中，上述脸部姿态可以是基于预先训练好的脸部姿态检测网络确定的。脸部姿态检测网络可以是任意结构的神经网络，如多层卷积神经网络，其可学习脸部图像样本以及对应标注的脸部姿态之间的对应关系，这里可以标注的脸部姿态包括头部相对摄像头的转动方向、转动角度等信息。这样，在将脸部图像输入到训练完成的脸部姿态检测网络，即可确定有关人脸的脸部姿态，如，可以是脸部向左偏45°，在这种情况下，可以在左偏45°的情况下比照标准左偏人脸图像进行有关预设关键特征点的提取，操作简单。
本公开实施例中，不同的脸部姿态在图像中呈现出的不同脸部子区域的面积可能存在差异，相应的平滑区域的面积也存在差异，仍以脸部向左偏45°为例，这时左边脸颊区域在图像中面积很小甚至几乎不可见，对应的平滑区域的面积接近于0。
上述区域属性信息可以是与平滑区域的亮度、面积、所处位置等中的至少一个属性有关的信息。在本公开的实施例中，根据各平滑区域的上述区域属性可以计算各平滑区域的权重，或者进行筛选处理，在加权或筛选处理之后获得与生理状态有关的平滑区域的图像信息作为待提取信息；或者，在计算出权重之后，执行时域信号提取操作，对各平滑区域对应的时域信号进行加权得到的信号作为与生理状态有关的待提取信息。
在本公开的一些实施例中，在区域属性信息包括区域亮度的情况下，与生理状态有关的待提取信息包括至少一个感兴趣区域的图像信息，这时可以实现基于区域亮度，对至少一个平滑区域进行区域筛选，得到至少一个感兴趣区域，进而可以对至少一个感兴趣区域的图像信息进行生理状态信息提取，得到生理状态检测结果，如图2B所示，为本公开实施例所提供的第二种生理状态检测方法的流程示意图，本公开实施例提供的步骤S104，可以通过如下步骤S201来实现：
S201：从至少一个平滑区域中，确定区域亮度满足预设条件的至少一个感兴趣区域。
这里，上述实施例提供的S105，可以通过如下步骤S202来实现：
S202：对至少一个感兴趣区域的图像信息进行生理状态信息提取，得到生理状态检测结果。
在本公开的一些实施例中，基于图像亮度阈值实现有关感兴趣区域的筛选，可以通过如下步骤来实现：
步骤一、确定脸部图像对应的图像亮度阈值和每个平滑区域的区域图像亮度；
步骤二、根据图像亮度阈值和区域图像亮度，对至少一个平滑区域进行过滤，得到至少一个候选平滑区域；
步骤三、在至少一个候选平滑区域中，选取区域亮度满足预设亮度条件的至少一个感兴趣区域。
这里，首先可以基于图像亮度阈值对平滑区域进行过滤，而后再基于过滤得到的候选平滑区域选取亮度最佳的感兴趣区域。
为了实现有效的过滤，本公开实施例可以通过最大亮度这一图像亮度阈值来进行限定，也可以通过最小亮度这一图像亮度阈值来进行限定。前者可以将区域图像亮度超过最大亮度的第一预设比例的平滑区域滤除，例如，可以将区域图像亮度超过最大亮度的90％的平滑区域滤除，以滤除可能过曝的相关平滑区域；后者可以将区域图像亮度未达到最小亮度的第二预设比例的平滑区域滤除，例如，可以将区域图像亮度小于最小亮度的30％的平滑区域滤除，以滤除可能过暗的相关平滑区域。
这里，上文涉及的区域图像亮度为一数值；其中，该数值可以指代区域亮度的平均值，即将区域内所有像素的像素亮度值相加并取均值得到的数值，也可以指代区域内所有像素中像素亮度最大值，也可以指代区域内所有像素中像素亮度最小值，也可以是对区域内所有像素的像素亮度值按照事先设定好的计算方式进行计算，得到的数值。
在本公开的一些实施例中，最大亮度和最小亮度可以是预设的亮度值，而针对不同的应用场景和/或不同的图像格式可以设置不同的亮度值，在实际应用中，可以根据需要来调整。这里，考虑到在一定的亮度范围内，亮度越高的区域具备更清晰的图像特征的可能性越大，这样所得到的生理检测结果也将更为准确，基于此，本公开实施例中可以选取亮度最大的候选平滑区域作为平滑区域。
本公开实施例中，在确定出的平滑区域为多个的情况下，可以首先确定每个平滑区域的亮度权重，而后基于亮度权重对多个平滑区域的生理状态信息提取结果进行融合。其中，有关亮度权重可以是基于对应区域的区域图像亮度来确定的，例如，对于区域图像亮度更大的平滑区域可以对应更大的亮度权重，反之，对于区域图像亮度更小的平滑区域可以对应更小的亮度权重，从而可以凸显亮度较大的平滑区域对于生理状态检测的影响，而弱化亮度较小的平滑区域对于生理状态检测的影响，这样所融合得到的生理状态检测结果将更为准确。
在本公开的一些实施例中，考虑到不同的平滑区域可能会受到不同的外部因素(如光照等)的影响，而导致各自所体现的由生理状态影响的图像特征并不相同，这里，为了尽可能的捕捉到更为有效的脸部特征以实现更为准确的生理状态检测，可以在基于至少一个候选平滑区域确定区域亮度满足预设条件的至少一个感兴趣区域的情况下，基于感兴趣区域的图像信息进行生理状态信息提取，这里，感兴趣区域的图像信息包括三个不同颜色通道的像素亮度信息，进而对至少一个感兴趣区域的图像信息进行生理状态信息提取，得到目标对象的生理状态检测结果，即上述步骤S202可以通过如下步骤来实现：
步骤一、基于感兴趣区域的三个颜色通道的像素亮度信息，确定表征目标对象的生理状态的时域信号；
步骤二、对时域信号进行频域转换，得到表征目标对象的生理状态的频域信号；
步骤三、基于频域信号的峰值，确定目标对象的生理状态值。
在本公开的一些实施例中，考虑到生理状态直接影响了目标对象的血流变化，而血 流变化又影响图像的亮度变化，因而，这里首先可以确定感兴趣区域对应于红、绿、蓝三个颜色通道中每一个颜色通道的时域亮度信号，形成RGB三维信号，然后对三个不同的颜色通道的时域亮度信号进行主成分分析，提取主成分(降维)后得到的一维信号作为表征目标对象的生理状态的时域信号，该时域信号可以是上述颜色通道中的其中一个通道(例如，绿色通道)的时域亮度信号确定的，且被选取的通道可以是最能表征血流变化的一个通道，即上述步骤一中关于“基于感兴趣区域的三个颜色通道的像素亮度信息，确定表征目标对象的生理状态的时域信号”，可以通过以下过程来实现：
首先，基于感兴趣区域对应的三个颜色通道的像素亮度信息，确定平滑区域对应于每一个颜色通道的时域亮度信号。
然后，对感兴趣区域对应于三个不同的颜色通道的时域亮度信号进行主成分分析，得到时域信号。
在一些实施例中，在确定每个感兴趣区域对应于每一个颜色通道的时域亮度信号的过程中，需要持续一段时间的视频流内的多帧脸部图像所对应的图像变化信息，这是考虑到诸如心率、呼吸频率、血氧、血压等生理状态信息往往需要一定时长的检测，且在确定感兴趣区域对应的时域亮度信号的过程中，可以利用多颜色通道的亮度值，以提升时域亮度信号提取的准确性。
这里，为了实现更为准确的主成分分析，在对三维时域亮度信号进行主成分分析之前，可以进行诸如正则化和Detrend滤波去噪等处理。除此之外，在主成分分析之后，还可以对得到的时域信号进行滑动平均滤波去噪处理，从而能够提升时域信号的精度，提升后续处理得到生理状态检测结果的准确度。
在本公开的一些实施例中，考虑到由于人脸与摄像头光轴存在夹角，不同平滑区域在摄像头成像所占的像素面积是存在差异的，因而这里可以通过选取的至少一个平滑区域，并通过平滑区域在画面中的像素面积进行加权，得到更接近实际情况的信号量，提升检测精度。
在确定脸部图像的平滑区域的情况下，本公开实施例提供的生理状态检测方法可以首先可以确定每一个平滑区域对应的时域亮度信号，而后基于各个平滑区域的面积对各个时域亮度信号进行加权处理，从而便于基于经过面积加权的时域亮度信号进行生理状态信息提取，这里所提取的生理状态检测结果可以是包括心率、呼吸频率、血氧、血压等中的至少一个在内的检测结果。进而在对每个平滑区域，确定对应的时域亮度信号的情况下，可以基于平滑区域的面积对确定的时域亮度信号进行加权处理，经过面积加权的时域亮度信号可以更大程度凸显脸部图像中较大的平滑区域对生理状态检测的影响，且能够弱化脸部图像中较小的平滑区域对生理状态检测的影响，进而能够提升检测的准确度，此外，整个检测过程无需专业设备的参与，可以随时随地进行测量，实用性更佳。
在本公开的一些实施例中，在区域属性信息包括区域面积的情况下，与生理状态有关的待提取信息可以包括经过加权的时域亮度信号，这时可实现基于区域面积，对至少一个平滑区域对应的时域亮度信号进行加权处理，得到经过面积加权的时域亮度信号，进而可以基于经过面积加权的时域亮度信号进行生理状态信息提取，得到生理状态检测结果，如图2C所示，为本公开实施例所提供的第三种生理状态检测方法的流程示意图，本公开实施例提供的S104，可以通过如下步骤203和步骤S204来实现：
S203：对每一个平滑区域，根据多帧脸部图像中平滑区域的至少一个颜色通道的像素亮度信息生成平滑区域对应的时域亮度信号；
S204：基于每个平滑区域的面积，对至少一个平滑区域对应的时域亮度信号进行加权处理，得到经过面积加权的时域亮度信号；
这里，上述实施例提供的S105，可以通过如下步骤S205来实现：
S205：基于经过面积加权的时域亮度信号进行生理状态信息提取，得到生理状态检 测结果。
在本公开的一些实施例中，在确定出信号表达能力比较强的各个平滑区域的情况下，这里可以针对每个平滑区域，基于多帧脸部图像中平滑区域对应于三个颜色通道的亮度值，确定平滑区域对应于每一个颜色通道的时域亮度信号。这样，针对每一个颜色通道，基于每个平滑区域的面积，对至少一个平滑区域在颜色通道下的时域亮度信号进行加权处理，得到在颜色通道下，经过面积加权的时域亮度信号，也即，本公开实施例实现的是各个颜色通道下的面积加权，使得每个颜色通道都具备了较强的血流变化表达能力。即本公开实施例提供的步骤S203，可以通过以下过程来实现：：
基于多帧脸部图像中平滑区域对应于三个颜色通道的像素亮度信息，确定平滑区域对应于每一个颜色通道的时域亮度信号。
对应地，上述实施例提供的步骤S204，可以通过以下过程来实现：
针对每一个颜色通道，基于每个平滑区域的面积，对至少一个平滑区域在颜色通道下的时域亮度信号进行加权处理，得到在颜色通道下，经过面积加权的时域亮度信号。
同时，上述实施例提供的步骤S205，可以通过以下过程来实现：
步骤一、对平滑区域在多个不同的颜色通道下，经过面积加权的时域亮度信号进行主成分分析，得到表征目标对象的生理状态的时域信号；
步骤二、对表征目标对象的生理状态的时域信号进行频域处理，并基于频域处理获得的频域信号的峰值，确定生理状态检测结果。
在一些实施例中，考虑到不同区域面积意味着对应平滑区域所包含的生理特征信息量也不同，所以这里，可以基于每个平滑区域的面积确定对应平滑区域的面积权重，并通过面积权重对至少一个平滑区域对应的时域亮度信号进行加权处理，以得到更接近实际情况的信号量，从而提升所生成生理状态检测结果的准确性。
本公开实施例中，不同的脸部姿态在图像中呈现出的不同脸部子区域的面积大小不同，所对应平滑区域的面积也不同，仍以脸部向左偏45°为例，这时左边脸颊区域在图像中几乎不可见，对应的平滑区域的面积为0。
在本公开的一些实施例中，还可以根据脸部姿态以及摄像设备的参数估计各个平滑区域的面积。例如，在通过神经网络等完成脸部姿态的检测之后，根据脸部姿态角及摄像头焦距等参数将正对摄像头的脸部模型投影至图像坐标系中，计算基于脸部关键点定位的各个平滑区域在图像坐标系中的边界点坐标，进而估算出各平滑区域在图像中的面积值。在脸部的部分区域亮度不足或部分关键点缺失的情况下，该面积值可以用于估算出图像中各个平滑区域的面积。本公开实施例中，可以基于各平滑区域对应的预设关键特征点的位置，确定平滑区域的面积。在实际应用中，可以将各个预设关键特征点的位置作为平滑区域的顶点位置，通过顶点距离确定平滑区域的长和宽，继而确定出区域面积，这样所确定出的区域面积更为准确。
对于区域面积更大的平滑区域而言，一定程度上可以承载更多的信息量，继而可以赋予更大的面积权重，反之，对于区域面积更小的平滑区域而言，一定程度上可以承载更少的信息量，继而可以赋予更小的面积权重。本公开实施例可以基于为各个区域面积所占总的区域面积的比例来确定对应各个平滑区域的面积权重，继而进行面积加权。
在进行面积加权之前，为了尽可能的确保有效信息量的提取，在确定出至少一个平滑区域的情况下，去除面积小于阈值的平滑区域。其中，阈值可以是预先设定的固定值，也可以是根据当前图像中整个人脸区域中各个平滑区域的面积分布情况确定的值。例如，可以将当前图像中各个平滑区域的面积的最大值的一定百分比(例如20％)作为该阈值。
在实际应用中，可以根据脸部姿态从平滑区域中去除图像中可见范围不满足预设的可见性要求的平滑区域，仍以脸部向左偏45°为例，这时右边脸颊区域面积小于预设的阈值，在图像中几乎不可见，这时可以直接去除对应的平滑区域。
在确定脸部图像的至少一个平滑区域的情况下，本公开实施例提供的生理状态检测方法中，在区域属性信息包括区域面积和区域亮度的情况下，与生理状态有关的待提取信息包括融合图像信息，这里的融合图像信息表示融合了多个维度的影像因素的图像信息，这时可以首先确定各平滑区域对生理状态检测结果的第一贡献度以及各平滑区域对生理状态检测结果的第二贡献度，其中第一贡献度表示各平滑区域内与区域面积相关的图像信息对生理状态检测结果的贡献程度，第二贡献度表示各平滑区域内与区域亮度相关的图像信息对生理状态检测结果的贡献程度，而后基于第一贡献度以及第二贡献度对至少一个平滑区域的图像信息进行融合，得到融合图像信息，最后基于融合图像信息进行生理状态信息提取，这里所提取的生理状态检测结果可以是包括心率、呼吸频率、血氧、血压等中的至少一个在内的检测结果；这样，可以根据各平滑区域的面积和像素亮度信息分别确定各平滑区域对生理状态检测结果的第一贡献度和第二贡献度的情况下，可以对至少一个平滑区域的图像信息进行融合，最后基于融合获得的图像信息进行生理状态信息提取，得到生理状态检测结果。这样，能够实现在进行生理状态检测的过程中，结合区域面积和区域亮度实现各平滑区域的图像信息的融合来进行，进而可以提升测量的准确度。如图2D所示，为本公开实施例所提供的第四种生理状态检测方法的流程示意图，即上述实施例提供的步骤S104，还可以通过以下步骤S206至步骤S208来实现：
S206：根据各平滑区域的面积，确定各平滑区域对生理状态检测结果的第一贡献度；
S207：根据各平滑区域的像素亮度信息，确定各平滑区域对生理状态检测结果的第二贡献度；
S208：基于第一贡献度和第二贡献度，对至少一个平滑区域的图像信息进行融合，得到融合图像信息。
这里，上述实施例提供的S105，可以通过如下步骤S209来实现：
S209：对融合图像信息进行生理状态信息提取，得到生理状态检测结果。
这里，有关融合过程可以是基于第一贡献度和第二贡献度进行加权融合，也即，针对一个平滑区域而言，在对应的第一贡献度和第二贡献度越高的情况下，该平滑区域所对应融合后的图像信息的表征能力也就越强。
或者，上述融合过程还可以基于第一贡献度和第二贡献度的比较结果来进行。在一些实施例中，若某一平滑区域的第一贡献度远大于第二贡献度，则可以选择基于该平滑区域的亮度信息来确定该平滑区域对应的生理状态信息提取结果，忽略面积的影响；若某一平滑区域的第一贡献度远小于第二贡献度，则可以仅基于该平滑区域的面积来确定该平滑区域对应的生理状态信息提取结果，忽略亮度的影响。
其中，有关第一贡献度可以是基于对应平滑区域的面积来确定，且第一贡献度值与对应平滑区域的面积呈正比，也即，面积越大的平滑区域一定程度上将为生理状态检测提供更高的第一贡献度，反之，面积越小的平滑区域为生理状态检测提供的第一贡献度将会更低，这是考虑到随着区域面积的增大，可以提取到的有效信息量也将随之提升，有效信息量的提升一定程度上可以提升生理状态检测的精度。
另外，有关第二贡献度可以是基于对应平滑区域的像素亮度信息来确定，第二贡献度与对应平滑区域的平均像素亮度值呈正比，也即，像素亮度越强的平滑区域一定程度上将为生理状态检测提供更高的第二贡献度，反之，像素亮度越弱的平滑区域为生理状态检测提供的第二贡献度将会更低，这是考虑到随着像素亮度的提升，对应的图像质量也将会随之提升，较好的图像质量一定程度上也可以提升生理状态检测的精度。除此之外，第二贡献度与对应平滑区域的像素亮度值的方差呈反比，在这种情况下，可以基于方差值来确定对应的第二贡献度。
在一些实施例中，在基于像素亮度信息确定第二贡献度时，需考虑可能出现的过曝或过暗的极端情况。在这些极端情况下，可能会影响生理状态检测的精度，基于此，在 实际应用中，可以先基于像素亮度对过曝或过暗的平滑区域进行滤除，而后再对符合亮度要求的平滑区域进行第二贡献度的确定，以确保最终生理状态检测结果的准确性。
在一些实施例中，在从每帧脸部图像中提取出平滑区域的情况下，一方面可以基于各平滑区域的面积确定各平滑区域对应的第一贡献度，另一方面可以基于各平滑区域的像素亮度信息确定各平滑区域对应的第二贡献度。
其中，在平滑区域的面积小于预设面积阈值的情况下，确定平滑区域的第一贡献度为0，也即，对于较小的平滑区域而言，其图像表征能力有限，这里可以不考虑对应的第一贡献度；另外，在根据各平滑区域的像素亮度信息确定各平滑区域的平均亮度值的情况下，在平均亮度值小于第一预设亮度阈值的情况下，确定平滑区域的第二贡献度为0，或在平均亮度值大于第二预设亮度阈值的情况下，确定平滑区域的第二贡献度为0，也即，对于可能过曝或过暗的平滑区域，这里可以不考虑对应的第二贡献度。
基于此，在一个平滑区域的面积属于正常面积，且该平滑区域处于正常亮度的情况下，即可分别确定对应面积权重和亮度权重。其中，面积权重可以按照如下步骤实现：
步骤一、将各平滑区域的面积进行求和运算，得到面积和值；
步骤二、针对每个平滑区域，计算平滑区域的面积与面积和值的比值，得到平滑区域的面积占比，作为平滑区域对应的面积权重。
这里，在对各平滑区域的面积进行求和后可以得到面积和值，对于平滑区域的面积在面积和值中占比越大的平滑区域可以确定出更大的面积权重，反之，对于平滑区域的面积在面积和值中占比越少的平滑区域可以确定出更小的面积权重。
另外，有关亮度权重可以按照如下步骤实现：
步骤一、将各平滑区域的平均亮度值进行求和运算，得到亮度和值；
步骤二、针对每个平滑区域，计算平滑区域的平均亮度值与亮度和值的比值，得到平滑区域的亮度占比，作为平滑区域对应的亮度权重。
这里，在对各平滑区域的平均亮度值进行求和后可以得到亮度和值，对于平滑区域的平均亮度值在亮度和值中占比越大的平滑区域可以确定出更大的亮度权重，反之，对于平滑区域的亮度在亮度和值中占比越少的平滑区域可以确定出更小的亮度权重。
在本公开的一些实施例中，基于面积权重和亮度权重可以对各平滑区域的图像信息进行融合以进行后续的生理状态检测，其中，第一贡献度包括面积权重，第二贡献度包括亮度权重；其中，有关融合过程，即上述步骤S208可以通过以下过程来实现：
步骤一，基于各平滑区域分别对应的面积权重和亮度权重的乘积，确定各平滑区域分别对应的权重总值；
步骤二，基于对应的权重总值对至少一个平滑区域的图像信息进行加权融合，确定融合图像信息。
这里，可以首先确定出每个平滑区域对应的权重总值，而后在为各个平滑区域的图像信息赋予对应的权重总值的情况下，通过加权求和即可确定融合图像信息。
在本公开的一些实施例中，对于面积权重和亮度权重均较大的平滑区域，其对应的权重总值也越大，一定程度上可以为融合图像信息提供更多的信息支撑，而对于面积权重较大且亮度权重较小的平滑区域、或者，面积权重较小且亮度权重较大的平滑区域，其对应的权重总值可能较大也可能较小，这需要基于图像分析结果来确定。通过加权融合获得的融合图像信息可以最大程度的挖掘出面部上有效的像素点，这为后续的生理状态检测提供了更多的数据支撑，从而有利于提升检测精度。
在本公开的一些实施例中，在确定出各平滑区域对生理状态检测结果的第一贡献度以及第二贡献度，且融合图像信息包括融合的时域亮度信号的情况下，本公开实施例提供的生理状态检测方法首先可以通过时域处理确定每一个平滑区域对应的时域亮度信号，而后基于各个平滑区域的第一贡献度和第二贡献度对各个时域亮度信号进行融合， 从而便于基于融合的时域亮度信号进行生理状态信息提取。
在一些实施例中，为了提升生理状态检测的准确度，这里，可以对融合的时域亮度信号进行频域处理，基于频域处理获得的频域信号可以分析出更多有用的信息，例如，可以确定各个频率成分的幅值分布和能量分布，从而得到关键幅度和能量分布的频率值。这里，可以基于频域信号的峰值确定目标对象的生理状态值。
本公开实施例中，在确定出融合图像信息的情况下，这里可以基于融合图像信息对应于三个颜色通道的亮度值，确定对应于每一个颜色通道的时域亮度信号。在对各个颜色通道进行主成分分析之后，即可以实现有关目标对象的生理状态检测，即上述步骤S208，还可以通过如下步骤来实现：
步骤一、针对每一平滑区域，对多帧图像中的平滑区域的亮度值进行时域处理，确定平滑区域对应的时域亮度信号；
步骤二、基于第一贡献度和第二贡献度，对多个平滑区域对应的时域亮度信号进行融合，得到融合的时域亮度信号。
这里，上述实施例提供的步骤S209可以通过以下过程来实现：
对融合的时域亮度信号进行频域处理，并基于频域处理获得的频域信号的峰值，确定生理状态结果。
在一些实施例中，对融合的时域亮度信号进行频域处理，可以指代对该融合的时域亮度信号进行主成分分析，得到对应的时域信号，进而对该时域信号进行频域转换，得到表征目标对象的生理状态的频域信号；最后基于频域信号的峰值，确定目标对象的生理状态结果。
在本公开的一些实施例中，在获取到视频流的情况下，可以先从视频流中提取目标对象的脸部图像，而后可以基于脸部图像查找目标对象的身体档案数据，同时可以基于脸部图像的图像信息进行时域处理和频域处理，得到目标对象的心率估计值，最后基于查找到的身体档案数据以及处理得到的心率估计值可以对目标对象的血压值进行估计，得到血压测量结果。这样，相比相关技术中需要借助专业的仪器或者设备所存在的测量不便的问题，本公开实施例提供的生理状态检测方法，可以确定用于表征血压波动情况的心率估计值，继而可以结合身体档案数据实现有关血压值的精确测量，测量结果更为精准，且无需借助专业设备，可随时随地进行实时地测量，实用性更佳。其中，血压(Blood Pressure，BP)是指血液在血管内流动时作用于单位面积血管壁的侧压力，它是推动血液在血管内流动的动力，是日常生活中一个非常重要的反应人体健康状态的生理指标。
实际应用中，考虑到心脏每跳动一下就会泵出血液，这些血液会沿着主动脉流向全身，而泵出的血液就是所测量到的血压，通常血压的高低与心脏泵血的力量有关，也即，人体的心跳情况与人体血压之间存在一定的联系，基于此，这里可以采用持续一段时间的视频流内的多帧脸部图像所对应的图像变化信息确定心率估计值，继而再结合身体档案数据来确定血压测量结果，这样所测量的血压值能够结合心跳情况以及不同人体的特定属性，从而更为符合实际场景的需要。
在本公开的一些实施例中，在目标图像的生理状态检测结果包括血压测量结果的情况下，上述实施例可以执行以下步骤S210至步骤S212，如图2E所示，为本公开实施例所提供的第五种生理状态检测方法的流程示意图:
S210：基于脸部图像，从预设档案库中查找与目标对象匹配的身体档案数据；
S211：基于多帧所述脸部图像的图像信息进行时域处理和频域处理，得到目标对象的心率估计值；
S212：基于身体档案数据和心率估计值，对目标对象的血压值进行估计，得到血压测量结果。
本公开实施例中，有关身体档案数据可以包括身高、体重、性别数据、年龄等数据， 这是考虑到不同的身高、体重、性别数据、年龄的人体所具备的身体机能情况并不相同，例如，小孩子的心跳普遍要比老人的心跳更块，在这里结合身体档案数据和心率估计值来估计血压值，是为了实现适应于各类人体的血压测量，更具适应性。
在本公开的一些实施例中，上述步骤S210可以通过至少之一的方式来实现：
方式一，基于脸部图像提取到的目标对象的脸部特征，从预设档案库中查找与目标对象的脸部特征对应的身体档案数据；
方式二，基于脸部图像识别到的目标对象的标识，从预设档案库中查找与标识对应的身体档案数据。
本公开实施例中，可以基于脸部图像从预设档案库中查找与目标对象匹配的身体档案数据，也即，各个对象的身体档案数据可以预先保存在预设档案库中，一旦获取到脸部图像，可以基于脸部图像或者从脸部图像中提取出的脸部特征从预设档案库中查找与目标对象对应的身体档案数据，操作简单。或者，一些身体档案数据也可以通过对目标对象的脸部图像进行分析来获得，如年龄、性别等。
在本公开的一些实施例中，可以基于脸部图像提取目标对象的脸部特征，而后从预设档案库中查找与目标对象的脸部特征对应的身体档案数据。其中，有关脸部特征可以是包括两眼间距、眉眼间距等在内的相关特征，或者可以是通过用于识别人脸的卷积神经网络等深度学习模型对人人脸图像进行特征提取所获得的特征表示，在预设档案库中可以预先记录了各个对象的脸部特征的情况下，基于所提取的脸部特征与预设档案库中的各个脸部特征之间的相似度，即可以查找出与目标对象对应的身体档案数据。
在本公开的一些实施例中，可以基于脸部图像识别目标对象，以确定目标对象的标识，而后从预设档案库中查找与标识对应的身体档案数据，作为与目标对象匹配的身体档案数据。也即，可以预先对各个对象设置不同的标识，在基于脸部图像进行人脸比对的情况下，可以确定对应标识的目标对象及其身体档案数据。
在本公开的一些实施例中，有关心率估计值可以是基于提取出的多帧脸部图像的图像信息进行时域处理和频域处理得到的，在实际应用中，可以基于多个颜色通道的图像信息的时域处理和频域处理得到更为符合实际需求的心率估计值，其实现过程可参考基于平滑区域的图像信息值进行生理状态信息提取，得到目标对象的生理状态检测结果，其中，该生理状态检测结果为心率估计值。
在本公开的一些实施例中，实现对目标对象的血压测量结果的过程，即上述步骤S212可以通过以下步骤来实现：
步骤一，根据射血时间与心率的线性拟合关系，确定与心率估计值对应的射血时间；以及，基于身体档案数据包括的性别数据、身高及体重，计算目标对象对应的人体表面积；以及，基于身体档案数据包括的体重和年龄、目标对象的心率估计值，确定目标对象的心脏弹性值；
步骤二，基于射血时间、人体表面积、身体档案数据包括的年龄以及心率估计值，确定目标对象对应的每搏输出量；
步骤三，基于目标对象对应的每搏输出量与心脏弹性值之间的比值，计算得到目标对象对应的脉压估计值；
步骤四，基于目标对象对应的脉压估计值以及身体档案数据包括的性别数据，计算得出目标对象的收缩压数值和舒张压数值。
在一些实施例中，结合对象的各个身体档案数据以及估计得到的心率估计值，可以实现包括收缩压数值和舒张压数值在内的血压测量值。为了便于理解上述血压测量的过程，下面为血压测量结果中的相关专业术语：
收缩压：心脏收缩将血液从心室泵入动脉时，血流对动脉壁产生的压力即为收缩压。
舒张压：当心脏舒张时，动脉血管壁具有一定的弹性而继续推动血流往前流动时的 压力就称为舒张压。
脉压：收缩压和舒张压之间的差值。
平均动脉压：一个心动周期中平均的动脉血压。
每博输出量：搏出量(Stroke Volume，SV)，是指一次心博中由一侧心室射出的血量。
射血时间：即左心室射血时间(Left Ventricular Ejection Time，LVET)，其定义为从主动脉瓣打开到主动脉瓣关闭的时间间隔，是左心室将血液射入主动脉的收缩期，可简称为(Ejection Time，ET)。
人体表面积(Body Surface Area，BSA)，许生文氏公式是符合人体特征的人体表面积公式。
心输出量(Cardiac Output，CO)：描述每单位时间内心脏所泵送的血液量。
体循环血管阻力(Systemic Vascular Resistance，SVR)。
在一些实施例中，可以结合具体的公式进行说明。如根据射血时间与心率的线性拟合关系，确定与心率估计值对应的射血时间，这里的线性拟合关系为ET＝364.5-1.23*HR；其中，ET用于表征射血时间，HR用于表征心率估计值，有关364.5以及1.23这些参数可以是根据特定数据集测试进行线性拟合的结果。
在一些实施例中，可以根据档案数据包括的性别数据、身高及体重，计算目标对象对应的人体表面积。这里，根据目标对象的健康档案信息(身高Height，体重Weight)和许生文氏公式，计算人体表面积BSA。
在一些实施例中，可以按照BSA＝0.0057*Height+0.0121*Weight+0.0882，确定目标对象为男性人体表面积；同时可以按照BSA＝0.0073*Height+0.0127*Weight-0.2106，确定目标对象为女性的人体表面积。
在确定出ET、HR、BSA的情况下，可以结合身体档案数据包括的年龄Age，确定目标对象对应的每搏输出量SV。可以按照如下公式(1)确定：
SV＝-6.6+0.25*(ET-35)-0.62*HR+40.4*BSA-0.51*Age    公式(1)；
基于目标对应的SV与心脏弹性值(Cardiac Elasticity，CE)之间的比值，可以计算得到目标对象对应的脉压估计值(Pulse pressure，Pp)，可以通过如下公式(2)来确定：
Pp＝SV/CE，      公式(2)；
其中，CE＝0.013*Weight-0.007*Age-0.004*HR+1.307，根据平均动脉压Pm＝CO*SVR计算Pm，其中CO为心输出量，SVR为体循环血管阻力。对于CO，男性平均为5L/min，女性平均为4.5L/min，SVR的经验值为18.31mmhgmin/L。
在确定出平均动脉压的情况下，可以计算收缩压(Ps＝Pm+2/3Pp)和舒张压(Pd＝Pm-1/3Pp)。这两个公式推导如公式(3)所示：
Pm＝Pd+1/3(Ps-Pd)   公式(3)；
再结合Pp的定义Pp＝Ps-Pd即可以推导出Ps和Pd的计算公式。
为了更好的监测目标对象的血压情况，可以将血压测量结果及多帧图像对应的采集时间存入目标对象的血压测量记录中，该血压测量记录可以存储目标对象在各个时段具有的血压测量情况，还可以基于一定预设时长内的血压测量记录生成对应的血压测量报告，从而便于对目标对象进行实时地监测，特别是在驾驶汽车的过程中，通过实时地血压监测可以降低由于血压过高或过低所导致的危险驾驶行为的现象，提高驾驶安全性。
有关血压测量报告可以是例如平滑5分钟内多次的测量结果，或者生成一段时间内血压动态变化报告(比如一天内多个不同时段的血压对比和变化趋势)，除此之外，还可以是其它维度对血压进行监测的相关报告情况，这里不做具体的限制。
在实际应用中，有关血压测量结果可以与目标对象的身体档案数据进行关联存储。随着年龄、体重等各种身体档案数据的变化，其血压测量结果也将随之发生变化，从而便于对目标对象进行更长时间跨度的身体状况分析，且能够在身体指标异常的情况下， 及时地进行提醒。
在实现血压测量之后，本公开实施例还可以展示血压测量结果以通过展示的血压测量结果为目标对象提供更好的车舱服务。
在一些实施例中，为了提升生理状态检测的准确度，这里，可以对时域信号进行频域转换，基于转换之后的频域信号可以分析出更多有用的信息，例如，可以确定各个频率成分的幅值分布和能量分布，从而得到关键幅度和能量分布的频率值。这里，可以基于频域信号的峰值确定目标对象的生理状态值。
以心率检测为例，这里可以确定频域信号的峰值pmax，通过pmax与心率基准值的求和结果可以得到原始心率测量值，其中，pmax表征的是心率变化量，心率基准值可以由基于经验的心率估计范围的下限来确定，还可以考虑诸如视频帧率、频域信号长度等因素的影响来调整心率基准值。
在确定心率之后，可以测算血氧饱和度和心率变异性等相关生理指标。针对血氧饱和度，可以利用红光(600至800nm)和近红光区域(800至1000nm)分别检测HbO2和Hb的时域信号，再计算相应的比值，就可得到血氧饱和度；针对心率变异性，在提取到时域信号后，通过计算每两个临近波峰的间距再结合帧率得到若干个间隔时间，并取这些间隔时间的标准差(Standard Deviation of NN Intervals，SDNN)，即得到心率变异性。
呼吸频率检测与心率检测的方法类似，其区别在于呼吸频率所在范围与心率所在范围不同，且对应的基准值设置不同，基于上述同样的方法可以实现呼吸频率检测。
本公开实施例实现的是多帧图像的生理状态检测，也即，多帧图像对应的图像变化信息可以表征出生理状态的变化情况。在实际应用中，有关视频流所确定的生理状态检测结果可以是随着图像帧的采集的持续而进行更新。
在本公开的一些实施例中，在获取到新的视频流包括的一帧或多帧图像的情况下，可以对新的视频流中的图像进行脸部检测，提取出车舱内的目标对象的脸部图像，而后确定每帧脸部图像中的至少一个平滑区域，并基于区域属性信息，对至少一个平滑区域进行加权处理或区域筛选，得到与生理状态有关的待提取信息，以及基于待提取信息对生理状态检测结果进行更新，若未达到预设检测时长，则再次基于获取的新的视频流进行更新，直至达到预设检测时长，得到更新后的生理状态检测结果。
这里，在获取到新的视频流包括的一帧或多帧图像的情况下，可以对新的视频流中的图像进行脸部检测，提取出车舱内的目标对象的脸部图像，而后确定脸部图像中的至少一个平滑区域，并从至少一个平滑区域中确定出图像亮度满足预设条件的至少一个感兴趣区域，这样即可以基于至少一个感兴趣区域的图像信息对生理状态检测结果进行更新，若未达到预设检测时长，则再次基于获取到的新的视频流进行更新，直至达到预设检测时长，得到更新后的生理状态检测结果。
在一些实施例中，在获取到新的视频流包括的一帧或多帧图像的情况下，可以对新的视频流中的图像进行脸部检测，提取出车舱内的目标对象的脸部图像，而后确定脸部图像中的至少一个平滑区域，并对每一个平滑区域，根据多帧脸部图像中平滑区域的至少一个颜色通道的像素亮度信息生成平滑区域对应的时域亮度信号，而后基于每个平滑区域的面积，对至少一个平滑区域对应的时域亮度信号进行加权处理，得到经过面积加权的时域亮度信号，并基于经过面积加权的时域亮度信号对生理状态检测结果进行更新，若未达到预设检测时长，则再次基于获取到的新的视频流进行更新，直至达到预设检测时长，得到更新后的生理状态检测结果。
在一些实施例中，在获取到新的视频流包括的一帧或多帧图像的情况下，可以对新的视频流中的图像进行脸部检测，提取出车舱内的目标对象的脸部图像，而后确定脸部图像中的至少一个平滑区域，并在确定各平滑区域对生理状态检测结果的第一贡献度和第二贡献度的情况下，基于第一贡献度和第二贡献度，对至少一个平滑区域的图像信息 进行融合，得到融合图像信息，并基于融合图像信息对生理状态检测结果进行更新，若未达到预设检测时长，则再次基于获取到的新的视频流进行更新，直至达到预设检测时长，得到更新后的生理状态检测结果。
这里仍以心率检测进行示例说明。在确定预设检测时长为30s的情况下，在30s内可以持续获取视频流。在基于起始视频流(例如起始的5秒内的视频流)的多帧图像计算出心率测量值的情况下，仍在30秒内。这时，随着图像帧的采集，图像帧数量增加，每增加一帧或者每增加n帧可以计算出一个新的心率测量值，然后通过滑动平均做平滑处理，到达30s后结束测量，得到最终测量结果。
作为示例，在车舱环境下，为了帮助目标对象进行更为快速的生理状态测量，这里，可以在一次生理状态检测过程中根据已获取的视频流的时长和预设检测时长生成用于提醒目标对象所需检测时长的检测进程提醒信号，例如，已获取视频流的时长(即当前的目标对象的生理状态检测已持续的检测时间)达到25秒，预设检测时长为30秒，则可以发出有关“请保持不动，还有5秒即可完成检测”的语音或屏幕提示；或者，在当前的目标对象的生理状态检测时长达到30秒时，发出“测量已完成”的语音或屏幕提示。
在实现生理状态检测之后，本公开实施例还可以展示生理状态检测结果以通过展示的生理状态检测为目标对象提供更好的车舱服务。
本公开实施例中，可以向车舱内的显示屏传送目标对象的生理状态检测结果，以在显示屏上进行显示，这样，车舱人员可以实时监测自身的生理状态情况下，还能够在自身的生理状态存在异常的情况下，及时就医或者采取其它必要的措施；同时还可以向生理状态检测应用的服务端传送目标对象的生理状态检测结果，以在目标对象通过生理状态检测应用请求获取检测结果的情况下，通过服务端向目标对象使用的终端设备发送生理状态检测结果。
也即，这里可以将目标对象的生理状态检测结果记录在服务端，在服务端还可以对理状态检测结果进行统计分析，例如，可以确定历史一个月、一周的生理状态统计结果，这样，在目标对象发起生理状态检测应用请求的情况下，可以将生理状态检测结果、统计结果等发送至目标对象的终端设备，以实现更为综合性的生理状态评估。
其中，上述生理状态检测应用可以是具体的用于进行生理状态检测的应用程序(Application，APP)，利用APP可以响应目标对象有关的检测结果的获取请求，继而实现在APP上的结果呈现，更具实用性。
本领域技术人员可以理解，在具体实施方式的上述方法中，各步骤的撰写顺序并不意味着严格的执行顺序而对实施过程构成任何限定，各步骤的具体执行顺序应当以其功能和可能的内在逻辑确定。
基于同一发明构思，本公开实施例中还提供了与生理状态检测方法对应的生理状态检测装置，由于本公开实施例中的装置解决问题的原理与本公开实施例上述生理状态检测方法相似，因此装置的实施可以参见方法的实施。
参照图3所示，为本公开实施例提供的一种生理状态检测装置的示意图，装置包括：获取部分301，被配置为获取摄像设备采集的视频流；提取部分302，被配置为从视频流中的多帧图像中提取目标对象的多帧脸部图像；确定部分303，被配置为确定脸部图像中的至少一个平滑区域；处理部分304，被配置为基于区域属性信息，对至少一个平滑区域进行加权处理或区域筛选，得到与生理状态有关的待提取信息；检测部分305，被配置为基于待提取信息进行生理状态信息提取，得到目标对象的生理状态检测结果。
在本公开的一些实施例中，确定部分303，还被配置为对脸部图像进行脸部特征点提取，得到脸部图像中的多个预设关键特征点的位置；基于多个预设关键特征点的位置，确定与多个预设关键特征点对应的脸部图像中的至少一个平滑区域。
在本公开的一些实施例中，区域属性信息包括区域亮度，与生理状态有关的待提取 信息包括至少一个感兴趣区域的图像信息，处理部分304，还被配置为从至少一个平滑区域中，确定区域亮度满足预设条件的至少一个感兴趣区域；检测部分305，还被配置为对至少一个感兴趣区域的图像信息进行生理状态信息提取，得到生理状态检测结果。
在本公开的一些实施例中，处理部分304，还被配置为确定脸部图像对应的图像亮度阈值和每个平滑区域的区域图像亮度；根据图像亮度阈值和区域图像亮度，对至少一个平滑区域进行过滤，得到至少一个候选平滑区域；在至少一个候选平滑区域中，选取区域亮度满足预设亮度条件的至少一个感兴趣区域。
在本公开的一些实施例中，感兴趣区域的图像信息包括三个不同颜色通道的像素亮度信息，检测部分305，还被配置为基于感兴趣区域的三个颜色通道的像素亮度信息，确定表征目标对象的生理状态的时域信号；对时域信号进行频域转换，得到表征目标对象的生理状态的频域信号；基于频域信号的峰值，确定目标对象的生理状态检测结果。
在本公开的一些实施例中，检测部分305，还被配置为基于感兴趣区域对应的三个颜色通道的像素亮度信息，确定感兴趣区域对应于每一个颜色通道的时域亮度信号；对感兴趣区域对应于三个不同的颜色通道的时域亮度信号进行主成分分析，得到时域信号。
在本公开的一些实施例中，区域属性信息包括区域面积，与生理状态有关的待提取信息包括经过加权的时域亮度信号，处理部分304，还被配置为对每一个平滑区域，根据多帧脸部图像中平滑区域的至少一个颜色通道的像素亮度信息生成平滑区域对应的时域亮度信号；基于每个平滑区域的面积，对至少一个平滑区域对应的时域亮度信号进行加权处理，得到经过面积加权的时域亮度信号；检测部分305，还被配置为基于经过面积加权的时域亮度信号进行生理状态信息提取，得到生理状态检测结果。
在本公开的一些实施例中，平滑区域对应于三个不同颜色通道的像素亮度信息，处理部分304，还被配置为基于多帧脸部图像中平滑区域对应于三个颜色通道的像素亮度信息，确定平滑区域对应于每一个颜色通道的时域亮度信号；针对每一个颜色通道，基于每个平滑区域的面积，对至少一个平滑区域在颜色通道下的时域亮度信号进行加权处理，得到在颜色通道下，经过面积加权的时域亮度信号；检测部分305，还被配置为对平滑区域在多个不同的颜色通道下，经过面积加权的时域亮度信号进行主成分分析，得到表征目标对象的生理状态的时域信号；对表征目标对象的生理状态的时域信号进行频域处理，并基于频域处理获得的频域信号的峰值，确定生理状态检测结果。
在本公开的一些实施例中，区域属性信息包括区域面积和区域亮度，与生理状态有关的待提取信息包括融合图像信息，处理部分304，还被配置为根据各平滑区域的面积，确定各平滑区域对生理状态检测结果的第一贡献度；根据各平滑区域的像素亮度信息，确定各平滑区域对生理状态检测结果的第二贡献度；基于第一贡献度和第二贡献度，对至少一个平滑区域的图像信息进行融合，得到融合图像信息；检测部分305，还被配置为对融合图像信息进行生理状态信息提取，得到生理状态检测结果。
在本公开的一些实施例中，第一贡献度包括面积权重，第二贡献度包括亮度权重，处理部分304，还被配置为基于各平滑区域分别对应的面积权重和亮度权重的乘积，确定各平滑区域分别对应的权重总值；基于对应的权重总值对至少一个平滑区域的图像信息进行加权融合，确定融合图像信息。
在本公开的一些实施例中，融合图像信息包括融合的时域亮度信号，处理部分304，还被配置为针对每一平滑区域，对多帧图像中的平滑区域的亮度值进行时域处理，确定平滑区域对应的时域亮度信号；基于第一贡献度和第二贡献度，对多个平滑区域对应的时域亮度信号进行融合，得到融合的时域亮度信号；检测部分305，被配置为对融合的时域亮度信号进行频域处理，并基于频域处理获得的频域信号的峰值，确定生理状态结果。
在本公开的一些实施例中，处理部分304，还被配置为基于脸部图像，从预设档案库中查找与目标对象匹配的身体档案数据；检测部分305，还被配置为基于多帧脸部图像的 图像信息进行时域处理和频域处理，得到目标对象的心率估计值；基于身体档案数据和心率估计值，对目标对象的血压值进行估计，得到血压测量结果。
在本公开的一些实施例中，处理部分304，还被配置为基于脸部图像提取到的目标对象的脸部特征，从预设档案库中查找与目标对象的脸部特征对应的身体档案数据；基于脸部图像识别到的目标对象的标识，从预设档案库中查找与标识对应的身体档案数据。
在本公开的一些实施例中，检测部分305，还被配置为根据射血时间与心率的线性拟合关系，确定与心率估计值对应的射血时间；以及，基于身体档案数据包括的性别数据、身高及体重，计算目标对象对应的人体表面积；以及，基于身体档案数据包括的体重和年龄、目标对象的心率估计值，确定目标对象的心脏弹性值；基于射血时间、人体表面积、身体档案数据包括的年龄以及心率估计值，确定目标对象对应的每搏输出量；基于目标对象对应的每搏输出量与心脏弹性值之间的比值，计算得到目标对象对应的脉压估计值；基于目标对象对应的脉压估计值以及身体档案数据包括的性别数据，计算得出所目标对象的收缩压数值和舒张压数值。
在本公开的一些实施例中，提取部分302，还被配置为根据视频流的脸部检测结果和指定的乘车位置，从检测出的脸部图像中确定出目标对象的脸部图像；其中，指定的乘车位置用于指示被测量的目标对象的位置。
在本公开的一些实施例中，该装置还包括：显示部分，被配置为向车舱内的显示屏传送目标对象的生理状态检测结果，以在显示屏上进行显示；发送部分，被配置为向生理状态测量应用的服务端传送目标对象的生理状态检测结果，以在目标对象通过生理状态应用请求获取检测结果的情况下，通过服务端向目标对象使用的终端设备发送生理状态检测结果。
在本公开的一些实施例中，检测部分304，还被配置为在获取到新的视频流的情况下，重复执行以下步骤，直至达到预设检测时长，得到更新后的生理状态检测结果：从新的视频流中提取目标对象的多帧脸部图像；确定每帧脸部图像中的至少一个平滑区域；以及基于区域属性信息，对至少一个平滑区域进行加权处理或区域筛选，得到与生理状态有关的待提取信息，并基于待提取图像信息对生理状态检测结果进行更新。
在本公开的一些实施例中，装置包括：提醒部分，被配置为根据已获取的视频流的时长和预设检测时长生成检测进程提醒信号，检测进程提醒信号用于提醒目标对象所需的检测时长。
关于装置中的各部分的处理流程、以及各部分之间的交互流程的描述可以参照上述方法实施例中的相关说明，这里不再详述。
本公开实施例还提供了一种电子设备，如图4所示，为本公开实施例提供的电子设备结构示意图，包括：处理器401、存储器402、和总线403。存储器402存储有处理器401可执行的机器可读指令(比如，图3中的装置中获取部分301、提取部分302、确定部分303、处理部分304以及检测部分305对应的执行指令等)，当电子设备运行时，处理器401与存储器402之间通过总线403通信，机器可读指令被处理器401执行时执行如下处理：
获取摄像设备采集的视频流；
从视频流中的多帧图像中提取目标对象的多帧脸部图像；
确定脸部图像中的至少一个平滑区域；
基于区域属性信息，对至少一个平滑区域进行加权处理或区域筛选，得到与生理状态有关的待提取信息；
基于待提取信息进行生理状态信息提取，得到目标对象的生理状态检测结果。
本公开实施例还提供一种计算机可读存储介质，该计算机可读存储介质上存储有计算机程序，该计算机程序被处理器运行时执行上述方法实施例中所述的生理状态检测方 法。其中，该存储介质可以是易失性或非易失的计算机可读取存储介质。
本公开实施例还提供一种计算机程序产品，该计算机程序产品承载有程序代码，所述程序代码包括的指令可用于执行上述方法实施例中所述的生理状态检测方法。
本公开实施例还提供一种计算机程序，该计算机程序包括计算机可读代码，在所述计算机可读代码在电子设备中运行的情况下，所述电子设备的处理器执行用于实现上述方法实施例中所述的生理状态检测方法。
其中，上述计算机程序产品可以具体通过硬件、软件或其结合的方式实现。在一个可选实施例中，所述计算机程序产品具体体现为计算机存储介质，在另一个可选实施例中，计算机程序产品具体体现为软件产品，例如软件开发包(Software Development Kit，SDK)等等。
所属领域的技术人员可以清楚地了解到，为描述的方便和简洁，上述描述的系统和装置的具体工作过程，可以参考前述方法实施例中的对应过程。在本公开所提供的几个实施例中，应该理解到，所揭露的系统、装置和方法，可以通过其它的方式实现。以上所描述的装置实施例仅仅是示意性的，例如，所述单元的划分，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式，又例如，多个单元或组件可以结合或者可以集成到另一个系统，或一些特征可以忽略，或不执行。另一点，所显示或讨论的相互之间的耦合或直接耦合或通信连接可以是通过一些通信接口，装置或单元的间接耦合或通信连接，可以是电性，机械或其它的形式。
作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是或者也可以不是物理单元，即可以位于一个地方，或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部单元来实现本实施例方案的目的。
另外，在本公开各个实施例中的各功能单元可以集成在一个处理单元中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个单元中。
所述功能如果以软件功能单元的形式实现并作为独立的产品销售或使用时，可以存储在一个处理器可执行的非易失的计算机可读取存储介质中。基于这样的理解，本公开的技术方案本质上或者说对现有技术做出贡献的部分或者该技术方案的部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质中，包括若干指令用以使得一台电子设备(可以是个人计算机，服务器，或者网络设备等)执行本公开各个实施例所述方法的全部或部分步骤。而前述的存储介质包括：U盘、移动硬盘、只读存储器(Read-Only Memory，ROM)、随机存取存储器(Random Access Memory，RAM)、磁碟或者光盘等各种可以存储程序代码的介质。
最后应说明的是：以上所述实施例，仅为本公开的具体实施方式，用以说明本公开的技术方案，而非对其限制，本公开的保护范围并不局限于此，尽管参照前述实施例对本公开进行了详细的说明，本领域的普通技术人员应当理解：任何熟悉本技术领域的技术人员在本公开揭露的技术范围内，其依然可以对前述实施例所记载的技术方案进行修改或可轻易想到变化，或者对其中部分技术特征进行等同替换；而这些修改、变化或者替换，并不使相应技术方案的本质脱离本公开实施例技术方案的精神和范围，都应涵盖在本公开的保护范围之内。因此，本公开的保护范围应所述以权利要求的保护范围为准。
工业实用性
本公开实施例提供了一种生理状态检测方法、装置、电子设备、存储介质及程序，其中，该方法包括：获取摄像设备采集的视频流；从视频流中的多帧图像中提取目标对象的多帧脸部图像；确定所述脸部图像中的至少一个平滑区域；基于区域属性信息，对所述至少一个平滑区域进行加权处理或区域筛选，得到与生理状态有关的待提取信息；对所述待提取信息进行生理状态信息提取，得到目标对象的生理状态检测结果。

Claims (22)

  1. 一种生理状态检测方法，包括：
    获取摄像设备采集的视频流；
    从所述视频流中的多帧图像中提取目标对象的多帧脸部图像；
    确定所述脸部图像中的至少一个平滑区域；
    基于区域属性信息，对所述至少一个平滑区域进行加权处理或区域筛选，得到与生理状态有关的待提取信息；
    对所述待提取信息进行生理状态信息提取，得到所述目标对象的生理状态检测结果。
  2. 根据权利要求1所述的方法，其中，所述确定所述脸部图像中的至少一个平滑区域，包括：
    对所述脸部图像进行脸部特征点提取，得到所述脸部图像中的多个预设关键特征点的位置；
    基于所述多个预设关键特征点的位置，确定与所述多个预设关键特征点对应的所述脸部图像中的至少一个平滑区域。
  3. 根据权利要求1或2所述的方法，其中，所述区域属性信息包括区域亮度，所述待提取信息包括至少一个感兴趣区域的图像信息，所述基于区域属性信息，对所述至少一个平滑区域进行加权处理或区域筛选，包括：
    从所述至少一个平滑区域中，确定所述区域亮度满足预设条件的至少一个感兴趣区域；
    所述对所述待提取信息进行生理状态信息提取，得到所述目标对象的生理状态检测结果，包括：
    对所述至少一个感兴趣区域的图像信息进行生理状态信息提取，得到所述生理状态检测结果。
  4. 根据权利要求3所述的方法，其中，所述从所述至少一个平滑区域中，确定所述区域亮度满足预设条件的至少一个感兴趣区域，包括：
    确定所述脸部图像对应的图像亮度阈值和每个平滑区域的区域图像亮度；
    根据所述图像亮度阈值和所述区域图像亮度，对所述至少一个平滑区域进行过滤，得到至少一个候选平滑区域；
    在所述至少一个候选平滑区域中，选取所述区域亮度满足预设亮度条件的所述至少一个感兴趣区域。
  5. 根据权利要求3或4所述的方法，其中，所述感兴趣区域的图像信息包括三个不同颜色通道的像素亮度信息，所述对所述至少一个感兴趣区域的图像信息进行生理状态信息提取，得到所述生理状态检测结果，包括：
    基于所述感兴趣区域的三个颜色通道的像素亮度信息，确定表征所述目标对象的生理状态的时域信号；
    对所述时域信号进行频域转换，得到表征所述目标对象的生理状态的频域信号；
    基于所述频域信号的峰值，确定所述目标对象的生理状态检测结果。
  6. 根据权利要求5所述的方法，其中，所述基于所述感兴趣区域的三个颜色通道的像素亮度信息，确定表征所述目标对象的生理状态的时域信号，包括：
    基于所述感兴趣区域对应的三个颜色通道的像素亮度信息，确定所述感兴趣区域对应于每一个所述颜色通道的时域亮度信号；
    对所述感兴趣区域对应于所述三个不同的颜色通道的时域亮度信号进行主成分分析，得到所述时域信号。
  7. 根据权利要求1或2所述的方法，其中，所述区域属性信息包括区域面积，所述待提取信息包括经过加权的时域亮度信号，所述基于区域属性信息，对所述至少一个平滑区域进行加权处理或区域筛选，得到与生理状态有关的待提取信息，包括：
    对每一个所述平滑区域，根据所述多帧脸部图像中所述平滑区域的至少一个颜色通道的像素亮度信息生成所述平滑区域对应的时域亮度信号；
    基于每个所述平滑区域的面积，对所述至少一个平滑区域对应的时域亮度信号进行加权处理，得到经过面积加权的时域亮度信号；
    所述对所述待提取信息进行生理状态信息提取，得到所述目标对象的生理状态检测结果，包括：
    基于所述经过面积加权的时域亮度信号进行生理状态信息提取，得到所述生理状态检测结果。
  8. 根据权利要求7所述的方法，其中，所述平滑区域对应于三个不同颜色通道的像素亮度信息，所述对每一个所述平滑区域，根据所述多帧脸部图像中所述平滑区域的至少一个颜色通道的像素亮度信息生成所述平滑区域对应的时域亮度信号，包括：
    基于所述多帧脸部图像中所述平滑区域对应于所述三个颜色通道的像素亮度信息，确定所述平滑区域对应于每一个所述颜色通道的时域亮度信号；
    所述基于每个所述平滑区域的面积，对所述至少一个平滑区域对应的时域亮度信号进行加权处理，得到经过面积加权的时域亮度信号，包括：
    针对每一个颜色通道，基于每个所述平滑区域的面积，对所述至少一个平滑区域在所述颜色通道下的所述时域亮度信号进行加权处理，得到在所述颜色通道下，经过面积加权的时域亮度信号；
    所述基于所述经过面积加权的时域亮度信号进行生理状态信息提取，得到所述生理状态检测结果，包括：
    对所述平滑区域在多个不同的颜色通道下，经过面积加权的时域亮度信号进行主成分分析，得到表征所述目标对象的生理状态的时域信号；
    对表征所述目标对象的生理状态的时域信号进行频域处理，并基于所述频域处理获得的频域信号的峰值，确定所述生理状态检测结果。
  9. 根据区权利要求1或2所述的方法，其中，所述区域属性信息包括区域面积和区域亮度，所述待提取信息包括融合图像信息，所述基于区域属性信息，对所述至少一个平滑区域进行加权处理或区域筛选，得到与生理状态有关的待提取信息，包括：
    根据各所述平滑区域的面积，确定各所述平滑区域对所述生理状态检测结果的第一贡献度；
    根据各所述平滑区域的像素亮度信息，确定各所述平滑区域对所述生理状态检测结果的第二贡献度；
    基于所述第一贡献度和所述第二贡献度，对所述至少一个平滑区域的图像信息进行融合，得到所述融合图像信息；
    所述对所述待提取信息进行生理状态信息提取，得到所述目标对象的生理状态检测结果，包括：
    对所述融合图像信息进行生理状态信息提取，得到所述生理状态检测结果。
  10. 根据权利要求9所述的方法，其中，所述第一贡献度包括面积权重，所述第二贡献度包括亮度权重，所述基于所述第一贡献度和所述第二贡献度，对所述至少一个平滑区域的图像信息进行融合，得到所述融合图像信息，包括：
    基于各所述平滑区域分别对应的面积权重和亮度权重的乘积，确定各所述平滑区域分别对应的权重总值；
    基于所述对应的权重总值对所述至少一个平滑区域的图像信息进行加权融合，确定 所述融合图像信息。
  11. 根据权利要求9所述的方法，其中，所述融合图像信息包括融合的时域亮度信号，所述基于所述第一贡献度和所述第二贡献度，对所述至少一个平滑区域的图像信息进行融合，得到所述融合图像信息，包括：
    针对每一平滑区域，对所述多帧图像中的所述平滑区域的亮度值进行时域处理，确定所述平滑区域对应的时域亮度信号；
    基于所述第一贡献度和所述第二贡献度，对多个所述平滑区域对应的时域亮度信号进行融合，得到所述融合的时域亮度信号；
    所述基于所述融合图像信息进行生理状态信息提取，得到所述生理状态检测结果，包括：
    对所述融合的时域亮度信号进行频域处理，并基于所述频域处理获得的频域信号的峰值，确定所述生理状态结果。
  12. 根据权利要求1至11任一项所述的方法，其中，所述方法还包括：
    基于所述脸部图像，从预设档案库中查找与所述目标对象匹配的身体档案数据；
    基于多帧所述脸部图像的图像信息进行时域处理和频域处理，得到所述目标对象的心率估计值；
    基于所述身体档案数据和所述心率估计值，对所述目标对象的血压值进行估计，得到血压测量结果。
  13. 根据权利要求12所述的方法，其中，所述基于所述脸部图像，从预设档案库中查找与所述目标对象匹配的身体档案数据，包括以下至少之一：
    基于所述脸部图像提取到的所述目标对象的脸部特征，从所述预设档案库中查找与所述目标对象的脸部特征对应的身体档案数据；
    基于所述脸部图像识别到的所述目标对象的标识，从所述预设档案库中查找与所述标识对应的身体档案数据。
  14. 根据权利要求12或13所述的方法，其中，所述基于所述身体档案数据和所述心率估计值，对所述目标对象的血压值进行估计，得到血压测量结果，包括：
    根据射血时间与心率的线性拟合关系，确定与所述心率估计值对应的射血时间；以及，基于所述身体档案数据包括的性别数据、身高及体重，计算所述目标对象对应的人体表面积；以及，基于所述身体档案数据包括的体重和年龄、所述目标对象的心率估计值，确定所述目标对象的心脏弹性值；
    基于所述射血时间、所述人体表面积、所述身体档案数据包括的年龄以及所述心率估计值，确定所述目标对象对应的每搏输出量；
    基于所述目标对象对应的每搏输出量与所述心脏弹性值之间的比值，计算得到所述目标对象对应的脉压估计值；
    基于所述目标对象对应的脉压估计值以及所述身体档案数据包括的性别数据，计算得出所述目标对象的收缩压数值和舒张压数值。
  15. 根据权利要求1至14任一所述的方法，其中，所述视频流包括车舱内的视频流，所述从所述视频流中的多帧图像中提取目标对象的多帧脸部图像，包括：
    根据所述视频流的脸部检测结果和指定的乘车位置，从检测出的脸部图像中确定出所述目标对象的所述脸部图像；其中，所述指定的乘车位置用于指示被测量的所述目标对象的位置。
  16. 根据权利要求15所述的方法，其中，所述方法还包括以下至少之一：
    向所述车舱内的显示屏传送所述目标对象的生理状态检测结果，以在所述显示屏上进行显示；
    向生理状态测量应用的服务端传送所述目标对象的生理状态检测结果，以在所述目 标对象通过所述生理状态应用请求获取所述检测结果的情况下，通过所述服务端向所述目标对象使用的终端设备发送所述生理状态检测结果。
  17. 根据权利要求1至16任一所述的方法，其中，所述方法还包括：
    在获取到新的视频流的情况下，重复执行以下步骤，直至达到预设检测时长，得到更新后的生理状态检测结果：
    从所述新的视频流中提取目标对象的多帧脸部图像；确定每帧所述脸部图像中的至少一个平滑区域；以及基于区域属性信息，对所述至少一个平滑区域进行加权处理或区域筛选，得到与生理状态有关的待提取信息，并基于所述待提取信息对所述生理状态检测结果进行更新。
  18. 根据权利要求17所述的方法，其中，所述方法还包括：
    根据已获取的视频流的时长和所述预设检测时长生成检测进程提醒信号，所述检测进程提醒信号用于提醒所述目标对象所需的检测时长。
  19. 一种生理状态检测装置，包括：
    获取部分，被配置为获取摄像设备采集的视频流；
    提取部分，被配置为从所述视频流中的多帧图像中提取目标对象的多帧脸部图像；
    确定部分，被配置为确定所述脸部图像中的至少一个平滑区域；
    处理部分，被配置为基于区域属性信息，对所述至少一个平滑区域进行加权处理或区域筛选，得到与生理状态有关的待提取信息；
    检测部分，被配置为对所述待提取信息进行生理状态信息提取，得到所述目标对象的生理状态检测结果。
  20. 一种电子设备，包括：处理器、存储器和总线，所述存储器存储有所述处理器可执行的机器可读指令，当电子设备运行时，所述处理器与所述存储器之间通过总线通信，所述机器可读指令被所述处理器执行时执行如权利要求1至18任一项所述的生理状态检测方法。
  21. 一种计算机可读存储介质，该计算机可读存储介质上存储有计算机程序，该计算机程序被处理器运行时执行如权利要求1至18任一项所述的生理状态检测方法。
  22. 一种计算机程序，所述计算机程序包括计算机可读代码，在所述计算机可读代码在电子设备中运行的情况下，所述电子设备的处理器执行用于实现如权利要求1至18任一项所述的生理状态检测方法。
PCT/CN2022/113755 2022-03-31 2022-08-19 生理状态检测方法、装置、电子设备、存储介质及程序 WO2023184832A1 (zh)

Applications Claiming Priority (8)

Application Number Priority Date Filing Date Title
CN202210346417.8 2022-03-31
CN202210344718.7A CN114863399A (zh) 2022-03-31 2022-03-31 一种生理状态检测方法、装置、电子设备及存储介质
CN202210344718.7 2022-03-31
CN202210344726.1A CN114648749A (zh) 2022-03-31 2022-03-31 一种生理状态检测方法、装置、电子设备及存储介质
CN202210346427.1A CN114708225A (zh) 2022-03-31 2022-03-31 一种血压测量方法、装置、电子设备及存储介质
CN202210346417.8A CN114663865A (zh) 2022-03-31 2022-03-31 一种生理状态检测方法、装置、电子设备及存储介质
CN202210346427.1 2022-03-31
CN202210344726.1 2022-03-31

Publications (1)

Publication Number Publication Date
WO2023184832A1 true WO2023184832A1 (zh) 2023-10-05

Family

ID=88198921

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/113755 WO2023184832A1 (zh) 2022-03-31 2022-08-19 生理状态检测方法、装置、电子设备、存储介质及程序

Country Status (1)

Country Link
WO (1) WO2023184832A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117668497A (zh) * 2024-01-31 2024-03-08 山西卓昇环保科技有限公司 基于深度学习实现环境保护下的碳排放分析方法及系统

Citations (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106236049A (zh) * 2016-10-12 2016-12-21 南京工程学院 基于视频图像的血压测量方法
WO2018179150A1 (ja) * 2017-03-29 2018-10-04 日本電気株式会社 心拍推定装置
JP2018164587A (ja) * 2017-03-28 2018-10-25 日本電気株式会社 脈波検出装置、脈波検出方法、及びプログラム
CN111275018A (zh) * 2020-03-06 2020-06-12 华东师范大学 一种基于环形感兴趣区域加权的非接触式心率信号提取方法
CN112819790A (zh) * 2021-02-02 2021-05-18 南京邮电大学 一种心率检测方法及装置
CN113197558A (zh) * 2021-03-26 2021-08-03 中南大学 心率与呼吸率检测方法、系统及计算机存储介质
CN113693573A (zh) * 2021-08-27 2021-11-26 西安电子科技大学 一种基于视频的非接触式多生理参数监测的系统及方法
CN114648749A (zh) * 2022-03-31 2022-06-21 上海商汤临港智能科技有限公司 一种生理状态检测方法、装置、电子设备及存储介质
CN114663865A (zh) * 2022-03-31 2022-06-24 上海商汤临港智能科技有限公司 一种生理状态检测方法、装置、电子设备及存储介质
CN114708225A (zh) * 2022-03-31 2022-07-05 上海商汤临港智能科技有限公司 一种血压测量方法、装置、电子设备及存储介质
CN114863399A (zh) * 2022-03-31 2022-08-05 上海商汤临港智能科技有限公司 一种生理状态检测方法、装置、电子设备及存储介质

Patent Citations (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106236049A (zh) * 2016-10-12 2016-12-21 南京工程学院 基于视频图像的血压测量方法
JP2018164587A (ja) * 2017-03-28 2018-10-25 日本電気株式会社 脈波検出装置、脈波検出方法、及びプログラム
WO2018179150A1 (ja) * 2017-03-29 2018-10-04 日本電気株式会社 心拍推定装置
CN111275018A (zh) * 2020-03-06 2020-06-12 华东师范大学 一种基于环形感兴趣区域加权的非接触式心率信号提取方法
CN112819790A (zh) * 2021-02-02 2021-05-18 南京邮电大学 一种心率检测方法及装置
CN113197558A (zh) * 2021-03-26 2021-08-03 中南大学 心率与呼吸率检测方法、系统及计算机存储介质
CN113693573A (zh) * 2021-08-27 2021-11-26 西安电子科技大学 一种基于视频的非接触式多生理参数监测的系统及方法
CN114648749A (zh) * 2022-03-31 2022-06-21 上海商汤临港智能科技有限公司 一种生理状态检测方法、装置、电子设备及存储介质
CN114663865A (zh) * 2022-03-31 2022-06-24 上海商汤临港智能科技有限公司 一种生理状态检测方法、装置、电子设备及存储介质
CN114708225A (zh) * 2022-03-31 2022-07-05 上海商汤临港智能科技有限公司 一种血压测量方法、装置、电子设备及存储介质
CN114863399A (zh) * 2022-03-31 2022-08-05 上海商汤临港智能科技有限公司 一种生理状态检测方法、装置、电子设备及存储介质

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117668497A (zh) * 2024-01-31 2024-03-08 山西卓昇环保科技有限公司 基于深度学习实现环境保护下的碳排放分析方法及系统
CN117668497B (zh) * 2024-01-31 2024-05-07 山西卓昇环保科技有限公司 基于深度学习实现环境保护下的碳排放分析方法及系统

Similar Documents

Publication Publication Date Title
Zhang et al. Driver drowsiness detection using multi-channel second order blind identifications
CN110276273B (zh) 融合面部特征与图像脉搏心率估计的驾驶员疲劳检测方法
Stricker et al. Non-contact video-based pulse rate measurement on a mobile service robot
JP6308161B2 (ja) 脈波検出装置、及び脈波検出プログラム
McDuff et al. Remote detection of photoplethysmographic systolic and diastolic peaks using a digital camera
KR101777738B1 (ko) 동영상을 이용하여 혈압을 추정하는 방법
US20110251493A1 (en) Method and system for measurement of physiological parameters
KR101738278B1 (ko) 영상을 이용한 감정 인식 방법
Rahman et al. Non-contact physiological parameters extraction using facial video considering illumination, motion, movement and vibration
Melchor Rodriguez et al. Video pulse rate variability analysis in stationary and motion conditions
CN110647815A (zh) 一种基于人脸视频图像的非接触式心率测量方法及系统
Casado et al. Face2PPG: An unsupervised pipeline for blood volume pulse extraction from faces
Bobbia et al. Remote photoplethysmography based on implicit living skin tissue segmentation
Fouad et al. Optimizing remote photoplethysmography using adaptive skin segmentation for real-time heart rate monitoring
CN114648749A (zh) 一种生理状态检测方法、装置、电子设备及存储介质
JP2014176584A (ja) 信号処理装置、信号処理方法及び信号処理プログラム
Park et al. Remote pulse rate measurement from near-infrared videos
WO2023184832A1 (zh) 生理状态检测方法、装置、电子设备、存储介质及程序
JP6455761B2 (ja) 脈波検出装置、及び脈波検出プログラム
Fernández et al. Unobtrusive health monitoring system using video-based physiological information and activity measurements
JP2004192552A (ja) 開閉眼判定装置
Oviyaa et al. Real time tracking of heart rate from facial video using webcam
CN114863399A (zh) 一种生理状态检测方法、装置、电子设备及存储介质
CN114663865A (zh) 一种生理状态检测方法、装置、电子设备及存储介质
CN114708225A (zh) 一种血压测量方法、装置、电子设备及存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22934652

Country of ref document: EP

Kind code of ref document: A1