US10685220B2 - Image detection method and image detection device for determining posture of a user - Google Patents

Image detection method and image detection device for determining posture of a user Download PDF

Info

Publication number
US10685220B2
US10685220B2 US15/904,237 US201815904237A US10685220B2 US 10685220 B2 US10685220 B2 US 10685220B2 US 201815904237 A US201815904237 A US 201815904237A US 10685220 B2 US10685220 B2 US 10685220B2
Authority
US
United States
Prior art keywords
user
image
predetermined value
pixel distribution
information
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active, expires
Application number
US15/904,237
Other versions
US20190122038A1 (en
Inventor
Yu-Ren Lai
Po Chun CHEN
Ching-An Cho
Kuo Ting HUANG
Yu-Yen Chen
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Wistron Corp
Original Assignee
Wistron Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Wistron Corp filed Critical Wistron Corp
Assigned to WISTRON CORP. reassignment WISTRON CORP. ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: CHEN, PO CHUN, CHEN, YU-YEN, CHO, CHING-AN, HUANG, KUO TING, LAI, YU-REN
Publication of US20190122038A1 publication Critical patent/US20190122038A1/en
Application granted granted Critical
Publication of US10685220B2 publication Critical patent/US10685220B2/en
Active legal-status Critical Current
Adjusted expiration legal-status Critical

Links

Images

Classifications

    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00Measuring for diagnostic purposes; Identification of persons
    • A61B5/72Signal processing specially adapted for physiological signals or for diagnostic purposes
    • A61B5/7235Details of waveform analysis
    • A61B5/7246Details of waveform analysis using correlation, e.g. template matching or determination of similarity
    • G06K9/00362
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00Measuring for diagnostic purposes; Identification of persons
    • A61B5/0033Features or image-related aspects of imaging apparatus classified in A61B5/00, e.g. for MRI, optical tomography or impedance tomography apparatus; arrangements of imaging apparatus in a room
    • A61B5/0037Performing a preliminary scan, e.g. a prescan for identifying a region of interest
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00Measuring for diagnostic purposes; Identification of persons
    • A61B5/103Detecting, measuring or recording devices for testing the shape, pattern, colour, size or movement of the body or parts thereof, for diagnostic purposes
    • A61B5/11Measuring movement of the entire body or parts thereof, e.g. head or hand tremor, mobility of a limb
    • A61B5/1116Determining posture transitions
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00Measuring for diagnostic purposes; Identification of persons
    • A61B5/48Other medical applications
    • A61B5/4806Sleep evaluation
    • G06K9/3233
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/20Image preprocessing
    • G06V10/25Determination of region of interest [ROI] or a volume of interest [VOI]
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/103Static body considered as a whole, e.g. static pedestrian or occupant recognition
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00Measuring for diagnostic purposes; Identification of persons
    • A61B5/103Detecting, measuring or recording devices for testing the shape, pattern, colour, size or movement of the body or parts thereof, for diagnostic purposes
    • A61B5/11Measuring movement of the entire body or parts thereof, e.g. head or hand tremor, mobility of a limb
    • A61B5/1121Determining geometric values, e.g. centre of rotation or angular range of movement
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00Measuring for diagnostic purposes; Identification of persons
    • A61B5/45For evaluating or diagnosing the musculoskeletal system or teeth
    • A61B5/4538Evaluating a particular part of the muscoloskeletal system or a particular medical condition
    • A61B5/4561Evaluating static posture, e.g. undesirable back curvature
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00Pattern recognition
    • G06F18/20Analysing
    • G06F18/24Classification techniques
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/0002Inspection of images, e.g. flaw detection
    • G06T7/0012Biomedical image inspection
    • G06T7/0014Biomedical image inspection using an image reference approach

Definitions

  • the disclosure generally relates to an image detection method and an image detection device, and more particularly, to an image detection method and an image detection device for determining the posture of a user.
  • Image detection technology can be utilized to determine the various postures of a user, such as the sleeping posture of an old person, in order to help care for the old person.
  • the facial features of a user may be used to determine sleeping posture, the facial features of a user are quite unstable.
  • the central portion of the face has more features, which result in a higher detection rate, and the side portion of the face has fewer features, which results in a lower detection rate. If the face of the user is almost completely covered, it becomes very hard to detect. Therefore, an image detection method and an image detection device are needed to improve the identification rate and the detection efficiency.
  • the invention proposes an image detection method and an image detection device.
  • a body feature such as the trunk or a limb of user is selected as the feature parameter for matching and analysis, and the pixel accumulation value of the user sketch is calculated to perform a pixel distribution analysis. Even if the face of the user cannot be identified, the sleeping posture of the user can be precisely determined based on the above body features, in order to help take care of the user.
  • an image detection method for determining the posture of a user includes obtaining a reference image of the user in a region of interest (ROI); obtaining a test image of the user in the ROI; executing a feature matching analysis of a feature parameter of the test image and a feature parameter of the reference image to obtain user similarity information of the test image and the reference image; executing a pixel distribution analysis for the test image to obtain user pixel distribution information; and determining the posture of the user according to the user similarity information and the user pixel distribution information.
  • ROI region of interest
  • the pixel distribution analysis is utilized to accumulate pixels for a user sketch of the test image along a first direction to obtain the user pixel distribution information.
  • the test image is different from the reference image.
  • the reference image is determined based on whether the pixel difference derived from subtracting two images is smaller than a predetermined value or not.
  • the ROI is a bed on which the user is lying, and the feature parameter comprises any one of the limbs, any one of the facial features, the trunk of the user, or a specific figure of a piece of clothing worn by the user.
  • the similarity information of the feature parameter of the test image and the feature parameter of the reference image is calculated to determine whether it is greater than a first predetermined value or not, and the pixel distribution analysis is utilized to calculate a pixel accumulation value of the user sketch along the first direction respectively based on a sequence along a second direction, and whether the user pixel distribution information of the pixel accumulation value is greater than a second predetermined value or not.
  • the first direction extends from the head of the user to the foot of the user, and the second direction is different from the first direction. In one embodiment, the second direction is vertical to the first direction.
  • the similarity information is greater than the first predetermined value and the user pixel distribution information is greater than the second predetermined value, it is determined that the user is in the supine position; and when the similarity information is greater than the first predetermined value and the user pixel distribution information is smaller than or equal to the second predetermined value, it is determined that the user is in a lateral recumbent position.
  • the similarity information is smaller than or equal to the first predetermined value and the user pixel distribution information is greater than the second predetermined value, it is determined that the user is in the prone position; and when the similarity information is smaller than the first predetermined value and the user pixel distribution information is smaller than or equal to the second predetermined value, it is determined that the user is in a lateral recumbent position.
  • an image detection device in another embodiment, includes a sensor and a processor.
  • the sensor is configured to detect a test image and a reference image of a user in a ROI.
  • the processor includes a feature matching module and a determination module.
  • the feature matching module is configured to execute a feature matching analysis of a feature parameter of the test image and a feature parameter of the reference image to obtain user similarity information of the test image and the reference image, and execute a pixel distribution analysis for the test image to obtain user pixel distribution information.
  • the determination module is configured to determine the posture of the user based on the user similarity information and the user pixel distribution information.
  • FIG. 1A is a schematic of an image detection device according to an embodiment of the invention.
  • FIG. 1B is a schematic of an image detection device according to another embodiment of the invention.
  • FIG. 1C is a schematic of a processor according to an embodiment of the invention.
  • FIG. 2 is a flow chart of an image detection method according to an embodiment of the invention.
  • FIG. 3A is a schematic of an image according to an embodiment of the invention.
  • FIG. 3B is a schematic of executing a secondary differentiation on the sketch of the image according to an embodiment of the invention.
  • FIGS. 4A, 4B and 4C are a flow chart of an image detection method according to another embodiment of the invention.
  • FIG. 5A is a schematic of calculating the user pixel distribution information of the pixel accumulation value according to an embodiment of the invention.
  • FIG. 5B is a schematic of calculating the user pixel distribution information of the pixel accumulation value according to another embodiment of the invention.
  • first and second features are formed in direct contact
  • additional features may be formed between the first and second features, such that the first and second features may not be in direct contact
  • present disclosure may repeat reference numerals and/or letters in the various examples. This repetition is for the purpose of simplicity and clarity and does not in itself dictate a relationship between the various embodiments and/or configurations discussed.
  • FIG. 1A is a schematic of an image detection device according to an embodiment of the invention.
  • the image detection device 10 includes a data storage device 100 , a processor 200 , a display device 300 , a communication device 400 , and a sensor 500 .
  • the image detection device 10 could be an independent electronic device, or it could be embedded in a mobile electronic device such as a cell phone, a tablet computer, a laptop computer, a gaming apparatus, an electronic book, or a PDA; or it could be an electronic device such as a desktop calculator, a server, a ticket machine, a digital picture frame, a point-of-sale terminal, or a digital map; or it could be any electronic device equipped with a touch module (such as a touch IC).
  • a touch module such as a touch IC
  • the data storage device 100 includes one or more of the following: a random access memory (RAM), a read-only memory (ROM), a flash memory, a hard disk, a soft disk, a magnetic memory, a compact disc (CD), or a digital video disk (DVD).
  • RAM random access memory
  • ROM read-only memory
  • flash memory a hard disk
  • hard disk a soft disk
  • magnetic memory a magnetic memory
  • CD compact disc
  • DVD digital video disk
  • the processor 200 of the image detection device 10 is coupled to the data storage device 100 to access the data in the data storage device 100 .
  • the processor 200 may include a digital signal processor (DSP), a microcontroller (MCU), a central-processing unit (CPU) or a plurality of parallel processors relating the parallel processing environment to implement the operating system (OS), firmware, driver and/or other applications of the electronic device.
  • the display device 300 is utilized to display data in the data storage device 100 .
  • the display 300 could be a projective display device, a 3D-image display device, an organic LED display, electronic paper, a system-integrated panel, an LED display liquid-crystal panel, or a touch display panel such as a resistive touch panel, a capacitive touch panel, an optical touch panel, or an electromagnetic touch panel.
  • a touch display panel such as a resistive touch panel, a capacitive touch panel, an optical touch panel, or an electromagnetic touch panel.
  • information about the postures is transmitted to the display 300 by the processor 200 so that the display 300 can display the various postures habitually used by the user.
  • the communication device 400 supports a wireless communication protocol in order to perform the data transmission with another electronic device.
  • the protocol for wireless communication could constitute GSM, GPRS, EDGE, UMTS, W-CDMA, CDMA2000, TD-CDMA, Bluetooth, NFC, WiFi, WiFi Direct, WiMAX, LTE, LTE-A or TD-LTE.
  • the sensor 500 is utilized to detect the image, sketch or figure of the user.
  • the sensor 500 is an optical sensor which receives an optical signal, transforms the optical signal into an electrical signal, and transmits the electrical signal to the processor 200 for calculation.
  • the sensor 500 may include APS, CMOS image sensor, CCD, infra-red sensor, optical-sensing transistor, or various optical cameras. Therefore, the image of the user could be detected by the sensor 500 even in a dark or twilight environment.
  • FIG. 1B is a schematic of an image detection device 10 according to another embodiment of the invention.
  • the image detection device 10 is designed separately and includes two image detection devices 10 A and 10 B.
  • the image detection device 10 A includes a sensor 500 and a communication device 400 A.
  • the image detection device 10 B includes a data storage device 100 , a processor 200 , a display 300 and a communication device 400 B.
  • the image detection device 10 A is installed in the environment of the user, and the image detection device 10 B is the main machine.
  • the image detection device 10 B could be the server, the large-scale machine or the cloud machine of the manufacture of the image detection device 10 .
  • the image detected by the sensor 500 of the image detection device 10 A is transmitted to the image detection device 10 B through the communication devices 400 A and 400 B for analysis.
  • FIG. 1C is a schematic of a processor 200 according to an embodiment of the invention.
  • the processor 200 includes a feature matching module 202 , a pixel distribution analysis module 204 and a determination module 206 .
  • the processor 200 further includes an indication module 208 .
  • the feature matching module 202 is utilized to perform the feature matching analysis on the feature parameter of the test image and the feature parameter of the reference image.
  • the pixel distribution analysis module 204 compares the feature parameter of the test image and the feature parameter of the reference image to obtain user similarity information of the test image and the reference image, and executes a pixel distribution analysis of the test image to obtain user pixel distribution information.
  • the determination module 206 determines the postures of the user based on the user similarity information and the user pixel distribution information.
  • the indication module 208 is utilized to generate sound or light to inform other people when the determination module 206 determines that the user is not well, so that the aforementioned other people can take care of the user.
  • the indication module 208 could be an alarm, a buzzer, a warning light, a flash, or an audio signal.
  • FIG. 2 is a flow chart of an image detection method according to an embodiment of the invention.
  • the processor 200 sets up the region of interest (ROI) for detection by the image detection device 10 .
  • the image detection device 10 is utilized to determine the sleeping posture of the user. Therefore, the ROI is the bed in which the user sleeps.
  • the ROI could be predetermined by the image detection device 10 , or determined by the user based on the environment of the user.
  • the sensor 500 starts to take pictures and obtains several images of the ROI. For example, the sensor 500 takes 20 ⁇ 60 images per second to record and detect the posture of users in the ROI.
  • the aforementioned number of images per second which are taken by the sensor 500 is merely for illustration, not for limitation.
  • the sensor 500 could adjust the number of images per second according to the user's specific environment and the user's specific needs. For example, when the user needs to improve the accuracy of determination, the number of images taken per second by the sensor 500 could be increased.
  • step S 202 the processor 200 determines whether the reference image of user in the ROI has been obtained or not. Because embodiments of the present invention are mainly utilized for detecting the sleeping posture of a user, it is required to detect a reference image of the user in a motionless state.
  • the processor 200 detects a first reference image
  • the first reference image is the matching reference for determining the posture of the user. If the reference image is not obtained, step S 202 will be executed. If the reference image is obtained, step S 203 will be executed so that the processor 200 determines the feature parameters of the user based on the reference image.
  • the feature parameter is any one of the limbs, any one of the facial features, the body of the user, or the specific pattern of the clothing worn by the user.
  • step S 204 the processor 200 determines whether the test image of the user in the ROI has been obtained or not.
  • the above test image is different from the reference image which is a matching reference. In another embodiment, the above test image is identical to the reference image which is a matching reference. If a test image is not obtained, then step S 204 will be executed. If the test image is obtained, step S 206 will be executed so that the processor 200 performs a feature matching analysis on the test image to compare the feature parameter of the test image and the feature parameter of the reference image for determining the similarity information of the test image and the reference image.
  • the above similarity information is related to the brightness of the user's environment.
  • step S 208 the processor 200 executes a pixel distribution analysis on the test image to calculate the extension of the sketch of the test image along the first direction for obtaining the user pixel distribution information.
  • the above first direction is defined as the extending direction from the head of the user to the foot of the user.
  • the processor 200 executes the feature matching analysis of step S 206 , and then executes the pixel distribution analysis of step S 208 .
  • the processor 200 executes the pixel distribution analysis of step S 208 , then executes the feature matching analysis of step S 206 . In another embodiment, the processor 200 simultaneously executes the pixel distribution analysis of step S 208 and the feature matching analysis of step S 206 , which is not limited by the present invention. Finally, in step S 210 , the postures of the user are determined according to the similarity information of user and the user pixel distribution information.
  • FIG. 3A is a schematic of an image P 1 according to an embodiment of the invention.
  • the image P 1 mainly includes the user and the bed in which the user sleeps (which is the ROI).
  • FIG. 3B is a schematic of executing a secondary differentiation on the sketch of the image according to an embodiment of the invention.
  • the secondary differentiation is utilized by the processor 200 to detect the user sketch.
  • the processor performs the differentiation on the user sketch more than two times.
  • the secondary differentiation is utilized to obtain obvious edge features of the user in order to detect the user sketch (such as the image P 1 ′ of FIG. 3B ).
  • the primary differentiation or the edge detection algorithm (such as the Sobel algorithm) is utilized by the processor 200 to obtain the detailed features of the sketch and edge.
  • the processor 200 can perform a principal component analysis (PCA).
  • PCA is a technique to analyze and simplify the database, reduce the degree of database, and maintain the database to contribute the greatest feature for the variables.
  • the processor 200 utilizes a PCA in association with the feature parameters such as the sketch, figure, body, or limbs of the user to analyze the body orientation of the user and the sleeping direction in the ROI. By utilizing the PCA, the body orientation of the user (which is the direction of the user on the bed) could be acquired to solve the problem of a distorted figure of the user.
  • FIGS. 4A, 4B and 4C are a flow chart of an image detection method according to another embodiment of the invention.
  • the ROI is set up by the processor 200 .
  • a plurality of images are obtained by the sensor 500 .
  • two images are subtracted by the processor 200 to obtain the pixel difference in order to determine whether there is moving vector or not.
  • various optical parameters such as the pixel, the resolution, and the color saturation could be extracted.
  • the processor 200 subtracts the successive first image and second image to obtain the pixel difference, and defines the pixel difference as the moving vector. If the pixel difference is greater than a predetermined value, it means there is a moving vector, and the processor 200 determines that the user is in an active state rather than a motionless state. If the pixel difference is less than or equal to the predetermined value, it means there is no moving vector, and the processor 200 determines that the user is in a motionless state.
  • the predetermined value is 200 . The value is for illustration, not for limiting the present invention. Specifically, the predetermined value could be adjusted based on the user environment and the need for the processor 200 to process an image.
  • step S 404 if the processor 200 determines there is a moving vector, step S 402 will be executed again. In other words, the processor 200 will abandon the first image and obtain the third image, and compare the second image with the third image to determine whether there is a moving vector or not. If the processor 200 determines that there is no moving vector, step S 406 will be executed to determine whether the user is lying down in the image or not. Therefore, by utilizing this method, a first reference image can be found in which the user has stopped moving and is lying still.
  • step S 406 the algorithm (such as an OpenCV algorithm) or the classifier (such as an SVM classifier) for detecting body features is utilized by the processor 200 .
  • the body features such as the sketch, figure, or skin color of the user, are utilized to determine whether the user is lying down within the image or not. If there is no body feature within the image, the processor 200 determines that the user is not lying down on the bed (or the ROI), and step S 402 will be executed again. If there is a body feature within the image, the processor 200 determines that the user is lying down on the bed, and step S 408 will be executed.
  • step S 408 the processor 200 determines whether the image is the first reference image or not. According to steps S 402 to S 408 , if the processor 200 determines that the image does not have a moving vector and the user is lying down on the bed, it means that the image is the first reference image. Afterwards, step S 410 is executed and the processor 200 determines feature parameters about the user. For example, a Hessian Matrix or Gabor Filter is executed to make the determination, but it is not limited thereto. The processor 200 utilizes a Hessian Matrix or Gabor Filter to extract the skin texture of the user to record the current state of the user.
  • the above feature parameter includes any one of the limbs, any one of the facial features, the body of the user, or the specific figure of the clothing worn by the user.
  • step S 412 the processor 200 performs the feature matching analysis on the test image to calculate the similarity information of the feature parameters of the test image and the feature parameters of the reference image.
  • step S 414 the processor 200 performs the pixel distribution analysis on the test image and utilizes the secondary differentiation to calculate the pixel accumulation value of the sketch along the first direction in the sequence of the second direction.
  • the second direction extends from the left side of the user to the right side of the user. The definition of the second direction is for illustration, not for limiting the present invention.
  • the processor 200 determines the sleeping posture of the user according to the feature matching analysis of step S 412 and the pixel distribution analysis of step S 414 .
  • the processor 200 performs a pixel distribution analysis on the test image to calculate the pixel accumulation value of the sketch along the first direction in the sequence of the second direction.
  • the first direction extends from the head of the user to the foot of the user, and the second direction is different from the first direction.
  • the second direction is not parallel to the first direction.
  • the second direction is perpendicular to the first direction, which is not limited by the present invention.
  • FIGS. 5A and 5B are schematics of calculating the user pixel distribution information of the pixel accumulation value according to an embodiment of the invention. As shown in the figures, images P 1 and P 2 both include the user and the ROI, which is the resting location of the user.
  • the Y axis is the first direction which extends from the head of the user to the foot of the user, and the X axis is the second direction.
  • the processor 200 calculates the pixel accumulation value on the Y axis, and compiles statistics for the distribution tendency for each pixel accumulation value on the X axis in order to obtain statistical diagrams HD 1 and HD 2 .
  • the pixel accumulation value is proportional to the weight of the body distribution.
  • the distribution of the statistical diagram HD 1 is more symmetrical. In other words, the pixel accumulation value of the central portion of the body is high, and the pixel accumulation value of the side portion of the body is low.
  • the distribution of the statistical diagram HD 2 is less symmetrical, which tends to indicate that the user is in a lateral recumbent position.
  • the processor 200 could utilize the PCA to obtain the body direction of the user, which is the direction in which the user is lying on the bed, and the problem of lying crooked can be solved.
  • the processor 200 could execute the feature matching analysis of step S 412 and the pixel distribution analysis of step S 414 simultaneously. In one embodiment, the processor 200 executes the feature matching analysis of step S 412 , then executes the pixel distribution analysis of step S 414 . In another embodiment, the processor 200 executes the pixel distribution analysis of step S 414 , then executes the feature matching analysis of step S 412 , which is not limited by the present invention.
  • step S 416 the processor 200 determines whether the similarity information is greater than the first predetermined value. For example, the processor 200 determines whether the similarity information of the feature parameters of the test image and the feature parameters of the reference image is greater than 80%.
  • the threshold value of 80% is used for illustration, not for limitation. The above threshold value could be adjusted by the processor 200 based on the user's environment and the image processing requirements. If yes, step S 418 will be executed. If not, step S 420 will be executed.
  • the processor 200 determines whether the user pixel distribution information of the above pixel accumulation value is greater than a second predetermined value or not. When the value of the user pixel distribution information is higher, it indicates that the distribution is more symmetrical.
  • step S 418 if the user pixel distribution information of the above pixel accumulation value is not greater than a second predetermined value, step S 422 will be executed and the processor 200 determines that the user is in a lateral recumbent position. If the user pixel distribution information of the above pixel accumulation value is greater than the second predetermined value, step S 424 will be executed and the processor 200 determines that the user is in the supine position.
  • step S 420 if the user pixel distribution information of the above pixel accumulation value is greater than the second predetermined value, step S 426 will be executed and the processor 200 determines that the user is in the prone position. If the user pixel distribution information of the above pixel accumulation value is not greater than the second predetermined value, step S 428 will be executed and the processor 200 determines that the user is in a lateral recumbent position.
  • a body feature such as the trunk or a limb of user is selected as the feature parameter for matching and analysis, and the pixel accumulation value of the user sketch is calculated to perform the pixel distribution analysis.
  • the sleeping posture of the user can still be determined based on the change of the covered figure and sketch. Even if the face of the user cannot be identified, the sleeping posture of the user can be precisely determined based on the above body features in order to help take care of the user.

Landscapes

  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Animal Behavior & Ethology (AREA)
  • General Health & Medical Sciences (AREA)
  • Veterinary Medicine (AREA)
  • Public Health (AREA)
  • Surgery (AREA)
  • Biophysics (AREA)
  • Pathology (AREA)
  • Biomedical Technology (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Medical Informatics (AREA)
  • Molecular Biology (AREA)
  • Multimedia (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Radiology & Medical Imaging (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Physiology (AREA)
  • Human Computer Interaction (AREA)
  • Signal Processing (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Psychiatry (AREA)
  • Artificial Intelligence (AREA)
  • Dentistry (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • Image Analysis (AREA)
  • Measurement Of The Respiration, Hearing Ability, Form, And Blood Characteristics Of Living Organisms (AREA)
  • Image Processing (AREA)
  • Measuring And Recording Apparatus For Diagnosis (AREA)

Abstract

An image detection method for determining the posture of a user includes: obtaining a reference image of the user in a region of interest (ROI); obtaining a test image of user at the ROI; executing a feature matching analysis of the test image which compares the feature parameter of the test image and the feature parameter of the reference image to determine the similarity information of the test image and the reference image; and executing a pixel distribution analysis of the test image to obtain user pixel distribution information; and determining the posture of the user based on the user similarity information and the user pixel distribution information.

Description

CROSS REFERENCE TO RELATED APPLICATIONS
This Application claims priority of Taiwan Patent Application No. 106136329 filed on Oct. 23, 2017, the entirety of which is incorporated by reference herein.
BACKGROUND OF THE INVENTION Field of the Invention
The disclosure generally relates to an image detection method and an image detection device, and more particularly, to an image detection method and an image detection device for determining the posture of a user.
Description of the Related Art
Image detection technology can be utilized to determine the various postures of a user, such as the sleeping posture of an old person, in order to help care for the old person. Although the facial features of a user may be used to determine sleeping posture, the facial features of a user are quite unstable. The central portion of the face has more features, which result in a higher detection rate, and the side portion of the face has fewer features, which results in a lower detection rate. If the face of the user is almost completely covered, it becomes very hard to detect. Therefore, an image detection method and an image detection device are needed to improve the identification rate and the detection efficiency.
BRIEF SUMMARY OF THE INVENTION
In order to solve the aforementioned problem, the invention proposes an image detection method and an image detection device. In embodiments of the present invention, a body feature such as the trunk or a limb of user is selected as the feature parameter for matching and analysis, and the pixel accumulation value of the user sketch is calculated to perform a pixel distribution analysis. Even if the face of the user cannot be identified, the sleeping posture of the user can be precisely determined based on the above body features, in order to help take care of the user.
In one embodiment of the invention, an image detection method for determining the posture of a user is provided. The image detection method includes obtaining a reference image of the user in a region of interest (ROI); obtaining a test image of the user in the ROI; executing a feature matching analysis of a feature parameter of the test image and a feature parameter of the reference image to obtain user similarity information of the test image and the reference image; executing a pixel distribution analysis for the test image to obtain user pixel distribution information; and determining the posture of the user according to the user similarity information and the user pixel distribution information.
The pixel distribution analysis is utilized to accumulate pixels for a user sketch of the test image along a first direction to obtain the user pixel distribution information. The test image is different from the reference image. The reference image is determined based on whether the pixel difference derived from subtracting two images is smaller than a predetermined value or not. The ROI is a bed on which the user is lying, and the feature parameter comprises any one of the limbs, any one of the facial features, the trunk of the user, or a specific figure of a piece of clothing worn by the user. The similarity information of the feature parameter of the test image and the feature parameter of the reference image is calculated to determine whether it is greater than a first predetermined value or not, and the pixel distribution analysis is utilized to calculate a pixel accumulation value of the user sketch along the first direction respectively based on a sequence along a second direction, and whether the user pixel distribution information of the pixel accumulation value is greater than a second predetermined value or not. The first direction extends from the head of the user to the foot of the user, and the second direction is different from the first direction. In one embodiment, the second direction is vertical to the first direction.
When the similarity information is greater than the first predetermined value and the user pixel distribution information is greater than the second predetermined value, it is determined that the user is in the supine position; and when the similarity information is greater than the first predetermined value and the user pixel distribution information is smaller than or equal to the second predetermined value, it is determined that the user is in a lateral recumbent position. When the similarity information is smaller than or equal to the first predetermined value and the user pixel distribution information is greater than the second predetermined value, it is determined that the user is in the prone position; and when the similarity information is smaller than the first predetermined value and the user pixel distribution information is smaller than or equal to the second predetermined value, it is determined that the user is in a lateral recumbent position.
In another embodiment of the invention, an image detection device is provided. The image detection device includes a sensor and a processor. The sensor is configured to detect a test image and a reference image of a user in a ROI. The processor includes a feature matching module and a determination module. The feature matching module is configured to execute a feature matching analysis of a feature parameter of the test image and a feature parameter of the reference image to obtain user similarity information of the test image and the reference image, and execute a pixel distribution analysis for the test image to obtain user pixel distribution information. The determination module is configured to determine the posture of the user based on the user similarity information and the user pixel distribution information.
BRIEF DESCRIPTION OF DRAWINGS
Aspects of the present disclosure are best understood from the following detailed description when read with the accompanying figures. It is noted that, in accordance with the standard practice in the industry, various features are not drawn to scale. In fact, the dimensions of the various features may be arbitrarily increased or reduced for clarity of discussion.
FIG. 1A is a schematic of an image detection device according to an embodiment of the invention;
FIG. 1B is a schematic of an image detection device according to another embodiment of the invention;
FIG. 1C is a schematic of a processor according to an embodiment of the invention;
FIG. 2 is a flow chart of an image detection method according to an embodiment of the invention;
FIG. 3A is a schematic of an image according to an embodiment of the invention;
FIG. 3B is a schematic of executing a secondary differentiation on the sketch of the image according to an embodiment of the invention;
FIGS. 4A, 4B and 4C are a flow chart of an image detection method according to another embodiment of the invention;
FIG. 5A is a schematic of calculating the user pixel distribution information of the pixel accumulation value according to an embodiment of the invention;
FIG. 5B is a schematic of calculating the user pixel distribution information of the pixel accumulation value according to another embodiment of the invention.
Corresponding numerals and symbols in the different figures generally refer to corresponding parts unless otherwise indicated. The figures are drawn to clearly illustrate the relevant aspects of the embodiments and are not necessarily drawn to scale.
DETAILED DESCRIPTION OF THE INVENTION
The following disclosure provides many different embodiments, or examples, for implementing different features of the provided subject matter. Specific examples of components and arrangements are described below to simplify the present disclosure. These are, of course, merely examples and are not intended to be limiting. For example, the formation of a first feature over or on a second feature in the description that follows may include embodiments in which the first and second features are formed in direct contact, and may also include embodiments in which additional features may be formed between the first and second features, such that the first and second features may not be in direct contact. In addition, the present disclosure may repeat reference numerals and/or letters in the various examples. This repetition is for the purpose of simplicity and clarity and does not in itself dictate a relationship between the various embodiments and/or configurations discussed.
FIG. 1A is a schematic of an image detection device according to an embodiment of the invention. The image detection device 10 includes a data storage device 100, a processor 200, a display device 300, a communication device 400, and a sensor 500. The image detection device 10 could be an independent electronic device, or it could be embedded in a mobile electronic device such as a cell phone, a tablet computer, a laptop computer, a gaming apparatus, an electronic book, or a PDA; or it could be an electronic device such as a desktop calculator, a server, a ticket machine, a digital picture frame, a point-of-sale terminal, or a digital map; or it could be any electronic device equipped with a touch module (such as a touch IC). The data storage device 100 includes one or more of the following: a random access memory (RAM), a read-only memory (ROM), a flash memory, a hard disk, a soft disk, a magnetic memory, a compact disc (CD), or a digital video disk (DVD).
In addition, the processor 200 of the image detection device 10 is coupled to the data storage device 100 to access the data in the data storage device 100. The processor 200 may include a digital signal processor (DSP), a microcontroller (MCU), a central-processing unit (CPU) or a plurality of parallel processors relating the parallel processing environment to implement the operating system (OS), firmware, driver and/or other applications of the electronic device. The display device 300 is utilized to display data in the data storage device 100. For example, the display 300 could be a projective display device, a 3D-image display device, an organic LED display, electronic paper, a system-integrated panel, an LED display liquid-crystal panel, or a touch display panel such as a resistive touch panel, a capacitive touch panel, an optical touch panel, or an electromagnetic touch panel. In one embodiment, when the various regular postures of the user have been determined by the processor 200, information about the postures is transmitted to the display 300 by the processor 200 so that the display 300 can display the various postures habitually used by the user.
The communication device 400 supports a wireless communication protocol in order to perform the data transmission with another electronic device. For example, the protocol for wireless communication could constitute GSM, GPRS, EDGE, UMTS, W-CDMA, CDMA2000, TD-CDMA, Bluetooth, NFC, WiFi, WiFi Direct, WiMAX, LTE, LTE-A or TD-LTE. The sensor 500 is utilized to detect the image, sketch or figure of the user. For example, the sensor 500 is an optical sensor which receives an optical signal, transforms the optical signal into an electrical signal, and transmits the electrical signal to the processor 200 for calculation. For example, the sensor 500 may include APS, CMOS image sensor, CCD, infra-red sensor, optical-sensing transistor, or various optical cameras. Therefore, the image of the user could be detected by the sensor 500 even in a dark or twilight environment.
FIG. 1B is a schematic of an image detection device 10 according to another embodiment of the invention. In the embodiment, the image detection device 10 is designed separately and includes two image detection devices 10A and 10B. The image detection device 10A includes a sensor 500 and a communication device 400A. The image detection device 10B includes a data storage device 100, a processor 200, a display 300 and a communication device 400B. Specifically, the image detection device 10A is installed in the environment of the user, and the image detection device 10B is the main machine. For example, the image detection device 10B could be the server, the large-scale machine or the cloud machine of the manufacture of the image detection device 10. The image detected by the sensor 500 of the image detection device 10A is transmitted to the image detection device 10B through the communication devices 400A and 400B for analysis.
FIG. 1C is a schematic of a processor 200 according to an embodiment of the invention. The processor 200 includes a feature matching module 202, a pixel distribution analysis module 204 and a determination module 206. The processor 200 further includes an indication module 208. The feature matching module 202 is utilized to perform the feature matching analysis on the feature parameter of the test image and the feature parameter of the reference image. The pixel distribution analysis module 204 compares the feature parameter of the test image and the feature parameter of the reference image to obtain user similarity information of the test image and the reference image, and executes a pixel distribution analysis of the test image to obtain user pixel distribution information. The determination module 206 determines the postures of the user based on the user similarity information and the user pixel distribution information. The indication module 208 is utilized to generate sound or light to inform other people when the determination module 206 determines that the user is not well, so that the aforementioned other people can take care of the user. For example, the indication module 208 could be an alarm, a buzzer, a warning light, a flash, or an audio signal.
FIG. 2 is a flow chart of an image detection method according to an embodiment of the invention. In step S200, the processor 200 sets up the region of interest (ROI) for detection by the image detection device 10. In one embodiment, the image detection device 10 is utilized to determine the sleeping posture of the user. Therefore, the ROI is the bed in which the user sleeps. The ROI could be predetermined by the image detection device 10, or determined by the user based on the environment of the user. When the setting is finished, in step S201, the sensor 500 starts to take pictures and obtains several images of the ROI. For example, the sensor 500 takes 20˜60 images per second to record and detect the posture of users in the ROI. The aforementioned number of images per second which are taken by the sensor 500 is merely for illustration, not for limitation. Specifically, the sensor 500 could adjust the number of images per second according to the user's specific environment and the user's specific needs. For example, when the user needs to improve the accuracy of determination, the number of images taken per second by the sensor 500 could be increased.
Afterwards, in step S202, the processor 200 determines whether the reference image of user in the ROI has been obtained or not. Because embodiments of the present invention are mainly utilized for detecting the sleeping posture of a user, it is required to detect a reference image of the user in a motionless state. When the processor 200 detects a first reference image, the first reference image is the matching reference for determining the posture of the user. If the reference image is not obtained, step S202 will be executed. If the reference image is obtained, step S203 will be executed so that the processor 200 determines the feature parameters of the user based on the reference image. In one embodiment, the feature parameter is any one of the limbs, any one of the facial features, the body of the user, or the specific pattern of the clothing worn by the user.
Afterwards, in step S204, the processor 200 determines whether the test image of the user in the ROI has been obtained or not. The above test image is different from the reference image which is a matching reference. In another embodiment, the above test image is identical to the reference image which is a matching reference. If a test image is not obtained, then step S204 will be executed. If the test image is obtained, step S206 will be executed so that the processor 200 performs a feature matching analysis on the test image to compare the feature parameter of the test image and the feature parameter of the reference image for determining the similarity information of the test image and the reference image. The above similarity information is related to the brightness of the user's environment. In a bright environment, it is determined to be similar when 70%˜90% of the test images and the reference images are the same. In a dark environment, it is determined to be similar when 30%˜50% of the test images and the reference images are the same. In step S208, the processor 200 executes a pixel distribution analysis on the test image to calculate the extension of the sketch of the test image along the first direction for obtaining the user pixel distribution information. The above first direction is defined as the extending direction from the head of the user to the foot of the user. Furthermore, it should be noted that, in the embodiment, the processor 200 executes the feature matching analysis of step S206, and then executes the pixel distribution analysis of step S208. In another embodiment, the processor 200 executes the pixel distribution analysis of step S208, then executes the feature matching analysis of step S206. In another embodiment, the processor 200 simultaneously executes the pixel distribution analysis of step S208 and the feature matching analysis of step S206, which is not limited by the present invention. Finally, in step S210, the postures of the user are determined according to the similarity information of user and the user pixel distribution information.
FIG. 3A is a schematic of an image P1 according to an embodiment of the invention. As shown in the figure, the image P1 mainly includes the user and the bed in which the user sleeps (which is the ROI). FIG. 3B is a schematic of executing a secondary differentiation on the sketch of the image according to an embodiment of the invention. In one embodiment, the secondary differentiation is utilized by the processor 200 to detect the user sketch. In other words, the processor performs the differentiation on the user sketch more than two times. Specifically, the secondary differentiation is utilized to obtain obvious edge features of the user in order to detect the user sketch (such as the image P1′ of FIG. 3B).
In another embodiment, the primary differentiation or the edge detection algorithm (such as the Sobel algorithm) is utilized by the processor 200 to obtain the detailed features of the sketch and edge. In addition, after obtaining the image P1′ of the user sketch, the processor 200 can perform a principal component analysis (PCA). PCA is a technique to analyze and simplify the database, reduce the degree of database, and maintain the database to contribute the greatest feature for the variables. In one embodiment, the processor 200 utilizes a PCA in association with the feature parameters such as the sketch, figure, body, or limbs of the user to analyze the body orientation of the user and the sleeping direction in the ROI. By utilizing the PCA, the body orientation of the user (which is the direction of the user on the bed) could be acquired to solve the problem of a distorted figure of the user.
FIGS. 4A, 4B and 4C are a flow chart of an image detection method according to another embodiment of the invention. In step S400, the ROI is set up by the processor 200. In step S402, a plurality of images are obtained by the sensor 500. Afterwards, in step S404, two images are subtracted by the processor 200 to obtain the pixel difference in order to determine whether there is moving vector or not. When the electric signals of several images are received by the processor 200, various optical parameters such as the pixel, the resolution, and the color saturation could be extracted.
In one embodiment, the processor 200 subtracts the successive first image and second image to obtain the pixel difference, and defines the pixel difference as the moving vector. If the pixel difference is greater than a predetermined value, it means there is a moving vector, and the processor 200 determines that the user is in an active state rather than a motionless state. If the pixel difference is less than or equal to the predetermined value, it means there is no moving vector, and the processor 200 determines that the user is in a motionless state. For example, the predetermined value is 200. The value is for illustration, not for limiting the present invention. Specifically, the predetermined value could be adjusted based on the user environment and the need for the processor 200 to process an image.
In step S404, if the processor 200 determines there is a moving vector, step S402 will be executed again. In other words, the processor 200 will abandon the first image and obtain the third image, and compare the second image with the third image to determine whether there is a moving vector or not. If the processor 200 determines that there is no moving vector, step S406 will be executed to determine whether the user is lying down in the image or not. Therefore, by utilizing this method, a first reference image can be found in which the user has stopped moving and is lying still.
Specifically, in step S406, the algorithm (such as an OpenCV algorithm) or the classifier (such as an SVM classifier) for detecting body features is utilized by the processor 200. The body features, such as the sketch, figure, or skin color of the user, are utilized to determine whether the user is lying down within the image or not. If there is no body feature within the image, the processor 200 determines that the user is not lying down on the bed (or the ROI), and step S402 will be executed again. If there is a body feature within the image, the processor 200 determines that the user is lying down on the bed, and step S408 will be executed.
In step S408, the processor 200 determines whether the image is the first reference image or not. According to steps S402 to S408, if the processor 200 determines that the image does not have a moving vector and the user is lying down on the bed, it means that the image is the first reference image. Afterwards, step S410 is executed and the processor 200 determines feature parameters about the user. For example, a Hessian Matrix or Gabor Filter is executed to make the determination, but it is not limited thereto. The processor 200 utilizes a Hessian Matrix or Gabor Filter to extract the skin texture of the user to record the current state of the user. The above feature parameter includes any one of the limbs, any one of the facial features, the body of the user, or the specific figure of the clothing worn by the user.
In addition, if the processor 200 determines that the image is not the first reference image, the feature matching analysis of step S412 and the pixel distribution analysis of step S414 will be executed. In step S412, the processor 200 performs the feature matching analysis on the test image to calculate the similarity information of the feature parameters of the test image and the feature parameters of the reference image. In step S414, the processor 200 performs the pixel distribution analysis on the test image and utilizes the secondary differentiation to calculate the pixel accumulation value of the sketch along the first direction in the sequence of the second direction. For example, the second direction extends from the left side of the user to the right side of the user. The definition of the second direction is for illustration, not for limiting the present invention. Afterwards, the processor 200 determines the sleeping posture of the user according to the feature matching analysis of step S412 and the pixel distribution analysis of step S414.
Specifically, during the pixel distribution analysis of step S414, the processor 200 performs a pixel distribution analysis on the test image to calculate the pixel accumulation value of the sketch along the first direction in the sequence of the second direction. The first direction extends from the head of the user to the foot of the user, and the second direction is different from the first direction. In one embodiment, the second direction is not parallel to the first direction. In another embodiment, the second direction is perpendicular to the first direction, which is not limited by the present invention. FIGS. 5A and 5B are schematics of calculating the user pixel distribution information of the pixel accumulation value according to an embodiment of the invention. As shown in the figures, images P1 and P2 both include the user and the ROI, which is the resting location of the user. The Y axis is the first direction which extends from the head of the user to the foot of the user, and the X axis is the second direction. Afterwards, the processor 200 calculates the pixel accumulation value on the Y axis, and compiles statistics for the distribution tendency for each pixel accumulation value on the X axis in order to obtain statistical diagrams HD1 and HD2.
Therefore, it could be realized that the pixel accumulation value is proportional to the weight of the body distribution. As shown in FIG. 5A, because the user in image P1 is lying on his or her back, the distribution of the statistical diagram HD1 is more symmetrical. In other words, the pixel accumulation value of the central portion of the body is high, and the pixel accumulation value of the side portion of the body is low. In the embodiment of FIG. 5B, because the user in image P2 is lying on his or her side in a lateral recumbent position, the distribution of the statistical diagram HD2 is less symmetrical, which tends to indicate that the user is in a lateral recumbent position. In other words, as for the statistical diagram HD2, there is higher pixel accumulation value on the side on which the user is lying. It should be noted that, in the embodiment of FIG. 5B, the side on which the user is lying results in a change of the body axis. Therefore, the processor 200 could utilize the PCA to obtain the body direction of the user, which is the direction in which the user is lying on the bed, and the problem of lying crooked can be solved.
In addition, it should be noted that the processor 200 could execute the feature matching analysis of step S412 and the pixel distribution analysis of step S414 simultaneously. In one embodiment, the processor 200 executes the feature matching analysis of step S412, then executes the pixel distribution analysis of step S414. In another embodiment, the processor 200 executes the pixel distribution analysis of step S414, then executes the feature matching analysis of step S412, which is not limited by the present invention.
Afterwards, in step S416, the processor 200 determines whether the similarity information is greater than the first predetermined value. For example, the processor 200 determines whether the similarity information of the feature parameters of the test image and the feature parameters of the reference image is greater than 80%. The threshold value of 80% is used for illustration, not for limitation. The above threshold value could be adjusted by the processor 200 based on the user's environment and the image processing requirements. If yes, step S418 will be executed. If not, step S420 will be executed.
During steps S418 and S420, the processor 200 determines whether the user pixel distribution information of the above pixel accumulation value is greater than a second predetermined value or not. When the value of the user pixel distribution information is higher, it indicates that the distribution is more symmetrical. In step S418, if the user pixel distribution information of the above pixel accumulation value is not greater than a second predetermined value, step S422 will be executed and the processor 200 determines that the user is in a lateral recumbent position. If the user pixel distribution information of the above pixel accumulation value is greater than the second predetermined value, step S424 will be executed and the processor 200 determines that the user is in the supine position.
Furthermore, in step S420, if the user pixel distribution information of the above pixel accumulation value is greater than the second predetermined value, step S426 will be executed and the processor 200 determines that the user is in the prone position. If the user pixel distribution information of the above pixel accumulation value is not greater than the second predetermined value, step S428 will be executed and the processor 200 determines that the user is in a lateral recumbent position.
In embodiments of the present invention, a body feature such as the trunk or a limb of user is selected as the feature parameter for matching and analysis, and the pixel accumulation value of the user sketch is calculated to perform the pixel distribution analysis. Even though the body of the user is covered by bedding or other cloth, the sleeping posture of the user can still be determined based on the change of the covered figure and sketch. Even if the face of the user cannot be identified, the sleeping posture of the user can be precisely determined based on the above body features in order to help take care of the user.
Use of ordinal terms such as “first”, “second”, “third”, etc., in the claims to modify a claim element does not by itself connote any priority, precedence, or order of one claim element over another or the temporal order in which acts of a method are performed, but are used merely as labels to distinguish one claim element having a certain name from another element having the same name (but for use of the ordinal term) to distinguish the claim elements.
While the invention has been described by way of example and in terms of the preferred embodiments, it should be understood that the invention is not limited to the disclosed embodiments. On the contrary, it is intended to cover various modifications and similar arrangements (as would be apparent to those skilled in the art). Therefore, the scope of the appended claims should be accorded the broadest interpretation so as to encompass all such modifications and similar arrangements.

Claims (13)

What is claimed is:
1. An image detection method for determining a posture of a user, comprising:
obtaining a reference image of a user in a region of interest (ROI);
obtaining a test image of the user in the ROI;
executing a feature matching analysis on a feature parameter of the test image and a feature parameter of the reference image to obtain user similarity information of the test image and the reference image;
executing a pixel distribution analysis for the test image to obtain user pixel distribution information; and
determining the posture of the user based on the user similarity information and the user pixel distribution information,
wherein the similarity information of the feature parameter of the test image and the feature parameter of the reference image is calculated to determine whether it is greater than a first predetermined value or not, and the pixel distribution analysis is utilized to calculate a pixel accumulation value of a user outline along the first direction respectively based on a sequence of a second direction, and whether the user pixel distribution information of the pixel accumulation value is greater than a second predetermined value or not, and
when the similarity information is greater than the first predetermined value and the user pixel distribution information is greater than the second predetermined value, it is determined that the user is in a supine position; and
when the similarity information is greater than the first predetermined value and the user pixel distribution information is smaller than or equal to the second predetermined value, it is determined that the user is in a lateral recumbent position.
2. The image detection method as claimed in claim 1, wherein the pixel distribution analysis is utilized to accumulate pixels for the user outline of the test image along a first direction to obtain the user pixel distribution information.
3. The image detection method as claimed in claim 1, wherein the test image is different from the reference image.
4. The image detection method as claimed in claim 1, further comprising:
determining the reference image from a plurality of images captured by an image sensor based on whether a pixel difference derived from subtracting two successive images among the images captured by the image sensor is smaller than a predetermined value or not.
5. The image detection method as claimed in claim 1, wherein the ROI is a bed on which the user is lying, and the feature parameter comprises any one of limbs, any one of facial features, trunk of the user, or a specific pattern of an article of clothing worn by the user.
6. The image detection method as claimed in claim 1, wherein:
when the similarity information is smaller than or equal to the first predetermined value and the user pixel distribution information is greater than the second predetermined value, it is determined that the user is in a prone position; and
when the similarity information is smaller than the first predetermined value and the user pixel distribution information is smaller than or equal to the second predetermined value, it is determined that the user is in a lateral recumbent position.
7. The image detection method as claimed in claim 1, wherein the first direction extends from head of the user to foot of the user, and the second direction is different from the first direction.
8. The image detection method as claimed in claim 7, wherein the second direction is vertical to the first direction.
9. An image detection device, comprising:
a sensor, configured to detect a test image and a reference image of a user in a region of interest (ROI); and
a processor, comprising:
a feature matching module, configured to execute a feature matching analysis of a feature parameter of the test image and a feature parameter of the reference image to obtain user similarity information of the test image and the reference image, and execute a pixel distribution analysis for the test image to obtain user pixel distribution information; and
a determination module, configured to determine a posture of the user based on the user similarity information and the user pixel distribution information,
wherein the processor calculates the similarity information of the feature parameter of the test image and the feature parameter of the reference image to determine whether it is greater than a first predetermined value or not, and the pixel distribution analysis is utilized to calculate a pixel accumulation value of a user outline along the first direction respectively based on a sequence of a second direction, and the processor calculates whether the user pixel distribution information of the pixel accumulation value is greater than a second predetermined value or not,
wherein when the similarity information is greater than the first predetermined value and the user pixel distribution information is greater than the second predetermined value, it is determined that the user is in a supine position,
wherein when the similarity information is greater than the first predetermined value and the user pixel distribution information is smaller than or equal to the second predetermined value, it is determined that the user is in a lateral recumbent position.
10. The image detection device as claimed in claim 9, wherein the pixel distribution analysis is utilized to accumulate pixels for the user outline of the test image along a first direction to obtain the user pixel distribution information.
11. The image detection device as claimed in claim 9, wherein:
when the similarity information is smaller than or equal to the first predetermined value and the user pixel distribution information is greater than the second predetermined value, it is determined that the user is in a prone position; and
when the similarity information is smaller than the first predetermined value and the user pixel distribution information is smaller than or equal to the second predetermined value, it is determined that the user is in a lateral recumbent position.
12. The image detection device as claimed in claim 9, wherein the first direction extends from the head of the user to the foot of the user, and the second direction is different from the first direction.
13. The image detection device as claimed in claim 12, wherein the second direction is vertical to the first direction.
US15/904,237 2017-10-23 2018-02-23 Image detection method and image detection device for determining posture of a user Active 2038-10-13 US10685220B2 (en)

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
TW106136329 2017-10-23
TW106136329A TWI647625B (en) 2017-10-23 2017-10-23 Image detection method and image detection device for determining postures of user
TW106136329A 2017-10-23

Publications (2)

Publication Number Publication Date
US20190122038A1 US20190122038A1 (en) 2019-04-25
US10685220B2 true US10685220B2 (en) 2020-06-16

Family

ID=65804132

Family Applications (1)

Application Number Title Priority Date Filing Date
US15/904,237 Active 2038-10-13 US10685220B2 (en) 2017-10-23 2018-02-23 Image detection method and image detection device for determining posture of a user

Country Status (3)

Country Link
US (1) US10685220B2 (en)
CN (1) CN109697400B (en)
TW (1) TWI647625B (en)

Families Citing this family (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110858286B (en) * 2018-08-23 2023-08-11 杭州海康威视数字技术股份有限公司 Image processing method and device for target recognition
EP3671530A1 (en) * 2018-12-17 2020-06-24 Koninklijke Philips N.V. Device, system and method for object recognition
WO2021146031A1 (en) * 2020-01-19 2021-07-22 Udisense Inc. Measurement calibration using patterned sheets
CN113645435A (en) * 2020-04-27 2021-11-12 财团法人工业技术研究院 Image monitoring device and method
USD937007S1 (en) 2020-07-30 2021-11-30 Udisense Inc. Bedsheet
CN115100600B (en) * 2022-06-30 2024-05-31 苏州市新方纬电子有限公司 Intelligent detection method and system for production line of battery pack
CN115239979B (en) * 2022-07-27 2024-08-23 中国人民解放军96901部队 Heterogeneous image matching method based on structural direction similarity

Citations (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20030236474A1 (en) 2002-06-24 2003-12-25 Balbir Singh Seizure and movement monitoring
TW200607350A (en) 2004-08-12 2006-02-16 Chunghwa Telecom Co Ltd Digital image monitoring system for motion detection
US20120026308A1 (en) 2010-07-29 2012-02-02 Careview Communications, Inc System and method for using a video monitoring system to prevent and manage decubitus ulcers in patients
CN103347438A (en) 2010-12-07 2013-10-09 Jt拉布斯有限公司 Sleep-posture sensing and monitoring system
TWM468728U (en) 2013-05-24 2013-12-21 Univ Central Taiwan Sci & Tech Real time human body poses identification system
US20140092247A1 (en) 2012-09-28 2014-04-03 Careview Communications, Inc. System and method for monitoring a fall state of a patient while minimizing false alarms
US20140163424A1 (en) * 2011-03-02 2014-06-12 Panasonic Corporation Posture estimation device, posture estimation system, and posture estimation method
TWI469765B (en) 2011-11-17 2015-01-21 Ind Tech Res Inst Apparatus and method of wireless measurement of sleep depth
TWI478695B (en) 2010-06-10 2015-04-01 Univ Nat Cheng Kung Sleeping efficiency analyzer and analyzing method thereof
CN205814297U (en) 2016-02-18 2016-12-21 深圳市云享智联科技有限公司 Sleep intelligent monitor system
CN107038402A (en) 2015-09-30 2017-08-11 富士通株式会社 The detection method and detection device of detection are moved for patient
US20190130178A1 (en) * 2016-03-30 2019-05-02 Equos Research Co., Ltd. Image recognition device and image recognition program

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
TWM361902U (en) * 2009-01-22 2009-08-01 Univ Lunghwa Sci & Technology Framework with human body posture identification function
JP2014518729A (en) * 2011-05-30 2014-08-07 コーニンクレッカ フィリップス エヌ ヴェ Apparatus and method for detection of body posture while sleeping
CN106056089B (en) * 2016-06-06 2019-06-11 中国科学院长春光学精密机械与物理研究所 A kind of 3 d pose recognition methods and system
CN106599792B (en) * 2016-11-23 2020-02-18 南京信息工程大学 Method for detecting hand driving violation behavior
CN107219925B (en) * 2017-05-27 2021-02-26 成都通甲优博科技有限责任公司 Posture detection method and device and server

Patent Citations (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20030236474A1 (en) 2002-06-24 2003-12-25 Balbir Singh Seizure and movement monitoring
TW200607350A (en) 2004-08-12 2006-02-16 Chunghwa Telecom Co Ltd Digital image monitoring system for motion detection
TWI478695B (en) 2010-06-10 2015-04-01 Univ Nat Cheng Kung Sleeping efficiency analyzer and analyzing method thereof
US20120026308A1 (en) 2010-07-29 2012-02-02 Careview Communications, Inc System and method for using a video monitoring system to prevent and manage decubitus ulcers in patients
CN103347438A (en) 2010-12-07 2013-10-09 Jt拉布斯有限公司 Sleep-posture sensing and monitoring system
US20140163424A1 (en) * 2011-03-02 2014-06-12 Panasonic Corporation Posture estimation device, posture estimation system, and posture estimation method
TWI469765B (en) 2011-11-17 2015-01-21 Ind Tech Res Inst Apparatus and method of wireless measurement of sleep depth
US20140092247A1 (en) 2012-09-28 2014-04-03 Careview Communications, Inc. System and method for monitoring a fall state of a patient while minimizing false alarms
TWM468728U (en) 2013-05-24 2013-12-21 Univ Central Taiwan Sci & Tech Real time human body poses identification system
CN107038402A (en) 2015-09-30 2017-08-11 富士通株式会社 The detection method and detection device of detection are moved for patient
CN205814297U (en) 2016-02-18 2016-12-21 深圳市云享智联科技有限公司 Sleep intelligent monitor system
US20190130178A1 (en) * 2016-03-30 2019-05-02 Equos Research Co., Ltd. Image recognition device and image recognition program

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
U.S. Appl. No. 15/920,412 to Chen et al., filed Mar. 13, 2018.

Also Published As

Publication number Publication date
TW201917638A (en) 2019-05-01
CN109697400A (en) 2019-04-30
CN109697400B (en) 2021-10-08
US20190122038A1 (en) 2019-04-25
TWI647625B (en) 2019-01-11

Similar Documents

Publication Publication Date Title
US10685220B2 (en) Image detection method and image detection device for determining posture of a user
US10699107B2 (en) Image detection method and image detection device for determining posture of a user
CN107992794B (en) A kind of biopsy method, device and storage medium
US9750420B1 (en) Facial feature selection for heart rate detection
KR102420100B1 (en) Electronic apparatus for providing health status information, method for controlling the same, and computer-readable storage medium
US10366487B2 (en) Electronic apparatus for providing health status information, method of controlling the same, and computer-readable storage medium
US8891819B2 (en) Line-of-sight detection apparatus and method thereof
KR102469720B1 (en) Electronic device and method for determining hyperemia grade of eye using the same
WO2019033569A1 (en) Eyeball movement analysis method, device and storage medium
US20160162746A1 (en) Detection devices and methods for detecting regions of interest
US20160371950A1 (en) Information processing apparatus, information processing method, and program
TWI680440B (en) Image detection method and image detection device for determining postures of user
KR101851099B1 (en) Method and System for Preventing Software-based Forward Head Posture Using Image Processing and Machine Learning
WO2019011073A1 (en) Human face live detection method and related product
JP2012038106A (en) Information processor, information processing method and program
JP6351243B2 (en) Image processing apparatus and image processing method
WO2020121425A1 (en) State determination device, state determination method, and state determination program
JP2012084012A (en) Image processing device, processing method therefor, and program
JP5482080B2 (en) Hand recognition device
US20190347786A1 (en) Method, apparatus, and electronic device having living body detection capability
JP6822326B2 (en) Watching support system and its control method
TWI676136B (en) Image detection method and image detection device utilizing dual analysis
US11257246B2 (en) Image detection method and image detection device for selecting representative image of user
JP2014191479A (en) Face recognition device, image pickup device, control method, and program
Ristiana et al. Vehicle Driver of Real-time Heart Rate Monitoring with Camera-Based Face Recognition

Legal Events

Date Code Title Description
FEPP Fee payment procedure

Free format text: ENTITY STATUS SET TO UNDISCOUNTED (ORIGINAL EVENT CODE: BIG.); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY

AS Assignment

Owner name: WISTRON CORP., TAIWAN

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:LAI, YU-REN;CHEN, PO CHUN;CHO, CHING-AN;AND OTHERS;REEL/FRAME:045126/0188

Effective date: 20180209

STPP Information on status: patent application and granting procedure in general

Free format text: DOCKETED NEW CASE - READY FOR EXAMINATION

STPP Information on status: patent application and granting procedure in general

Free format text: NON FINAL ACTION MAILED

STPP Information on status: patent application and granting procedure in general

Free format text: NOTICE OF ALLOWANCE MAILED -- APPLICATION RECEIVED IN OFFICE OF PUBLICATIONS

STPP Information on status: patent application and granting procedure in general

Free format text: PUBLICATIONS -- ISSUE FEE PAYMENT VERIFIED

STCF Information on status: patent grant

Free format text: PATENTED CASE

MAFP Maintenance fee payment

Free format text: PAYMENT OF MAINTENANCE FEE, 4TH YEAR, LARGE ENTITY (ORIGINAL EVENT CODE: M1551); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY

Year of fee payment: 4