WO2021060250A1 - 画像処理装置、画像処理方法及び画像処理プログラム - Google Patents
画像処理装置、画像処理方法及び画像処理プログラム Download PDFInfo
- Publication number
- WO2021060250A1 WO2021060250A1 PCT/JP2020/035722 JP2020035722W WO2021060250A1 WO 2021060250 A1 WO2021060250 A1 WO 2021060250A1 JP 2020035722 W JP2020035722 W JP 2020035722W WO 2021060250 A1 WO2021060250 A1 WO 2021060250A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- image
- image processing
- unit
- information
- comparison
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Ceased
Links
Images
Classifications
-
- A—HUMAN NECESSITIES
- A45—HAND OR TRAVELLING ARTICLES
- A45D—HAIRDRESSING OR SHAVING EQUIPMENT; EQUIPMENT FOR COSMETICS OR COSMETIC TREATMENTS, e.g. FOR MANICURING OR PEDICURING
- A45D44/00—Other cosmetic or toiletry articles, e.g. for hairdressers' rooms
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0012—Biomedical image inspection
- G06T7/0014—Biomedical image inspection using an image reference approach
- G06T7/0016—Biomedical image inspection using an image reference approach involving temporal comparison
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/168—Feature extraction; Face representation
- G06V40/171—Local features and components; Facial parts ; Occluding parts, e.g. glasses; Geometrical relationships
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/103—Measuring devices for testing the shape, pattern, colour, size or movement of the body or parts thereof, for diagnostic purposes
- A61B5/107—Measuring physical dimensions, e.g. size of the entire body or parts thereof
- A61B5/1072—Measuring physical dimensions, e.g. size of the entire body or parts thereof measuring distances on the body, e.g. measuring length, height or thickness
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/103—Measuring devices for testing the shape, pattern, colour, size or movement of the body or parts thereof, for diagnostic purposes
- A61B5/107—Measuring physical dimensions, e.g. size of the entire body or parts thereof
- A61B5/1079—Measuring physical dimensions, e.g. size of the entire body or parts thereof using optical or photographic means
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/48—Other medical applications
- A61B5/4848—Monitoring or testing the effects of treatment, e.g. of medication
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/48—Other medical applications
- A61B5/4887—Locating particular structures in or on the body
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/74—Details of notification to user or communication with user or patient; User input means
- A61B5/742—Details of notification to user or communication with user or patient; User input means using visual displays
- A61B5/743—Displaying an image simultaneously with additional graphical information, e.g. symbols, charts, function plots
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/74—Details of notification to user or communication with user or patient; User input means
- A61B5/742—Details of notification to user or communication with user or patient; User input means using visual displays
- A61B5/744—Displaying an avatar, e.g. an animated cartoon character
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61H—PHYSICAL THERAPY APPARATUS, e.g. DEVICES FOR LOCATING OR STIMULATING REFLEX POINTS IN THE BODY; ARTIFICIAL RESPIRATION; MASSAGE; BATHING DEVICES FOR SPECIAL THERAPEUTIC OR HYGIENIC PURPOSES OR SPECIFIC PARTS OF THE BODY
- A61H7/00—Devices for suction-kneading massage; Devices for massaging the skin by rubbing or brushing not otherwise provided for
- A61H7/007—Kneading
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G16H20/00—ICT specially adapted for therapies or health-improving plans, e.g. for handling prescriptions, for steering therapy or for monitoring patient compliance
- G16H20/70—ICT specially adapted for therapies or health-improving plans, e.g. for handling prescriptions, for steering therapy or for monitoring patient compliance relating to mental therapies, e.g. psychological therapy or autogenous training
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G16H30/00—ICT specially adapted for the handling or processing of medical images
- G16H30/20—ICT specially adapted for the handling or processing of medical images for handling medical images, e.g. DICOM, HL7 or PACS
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G16H30/00—ICT specially adapted for the handling or processing of medical images
- G16H30/40—ICT specially adapted for the handling or processing of medical images for processing medical images, e.g. editing
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G16H40/00—ICT specially adapted for the management or administration of healthcare resources or facilities; ICT specially adapted for the management or operation of medical equipment or devices
- G16H40/60—ICT specially adapted for the management or administration of healthcare resources or facilities; ICT specially adapted for the management or operation of medical equipment or devices for the operation of medical equipment or devices
- G16H40/67—ICT specially adapted for the management or administration of healthcare resources or facilities; ICT specially adapted for the management or operation of medical equipment or devices for the operation of medical equipment or devices for remote operation
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/0002—Remote monitoring of patients using telemetry, e.g. transmission of vital signals via a communication network
- A61B5/0015—Remote monitoring of patients using telemetry, e.g. transmission of vital signals via a communication network characterised by features of the telemetry system
- A61B5/0022—Monitoring a patient using a global network, e.g. telephone networks, internet
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/02—Detecting, measuring or recording for evaluating the cardiovascular system, e.g. pulse, heart rate, blood pressure or blood flow
- A61B5/021—Measuring pressure in heart or blood vessels
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/02—Detecting, measuring or recording for evaluating the cardiovascular system, e.g. pulse, heart rate, blood pressure or blood flow
- A61B5/024—Measuring pulse rate or heart rate
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/02—Detecting, measuring or recording for evaluating the cardiovascular system, e.g. pulse, heart rate, blood pressure or blood flow
- A61B5/026—Measuring blood flow
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61H—PHYSICAL THERAPY APPARATUS, e.g. DEVICES FOR LOCATING OR STIMULATING REFLEX POINTS IN THE BODY; ARTIFICIAL RESPIRATION; MASSAGE; BATHING DEVICES FOR SPECIAL THERAPEUTIC OR HYGIENIC PURPOSES OR SPECIFIC PARTS OF THE BODY
- A61H2201/00—Characteristics of apparatus not provided for in the preceding codes
- A61H2201/01—Constructive details
- A61H2201/0157—Constructive details portable
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61H—PHYSICAL THERAPY APPARATUS, e.g. DEVICES FOR LOCATING OR STIMULATING REFLEX POINTS IN THE BODY; ARTIFICIAL RESPIRATION; MASSAGE; BATHING DEVICES FOR SPECIAL THERAPEUTIC OR HYGIENIC PURPOSES OR SPECIFIC PARTS OF THE BODY
- A61H2201/00—Characteristics of apparatus not provided for in the preceding codes
- A61H2201/01—Constructive details
- A61H2201/0188—Illumination related features
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61H—PHYSICAL THERAPY APPARATUS, e.g. DEVICES FOR LOCATING OR STIMULATING REFLEX POINTS IN THE BODY; ARTIFICIAL RESPIRATION; MASSAGE; BATHING DEVICES FOR SPECIAL THERAPEUTIC OR HYGIENIC PURPOSES OR SPECIFIC PARTS OF THE BODY
- A61H2201/00—Characteristics of apparatus not provided for in the preceding codes
- A61H2201/12—Driving means
- A61H2201/1253—Driving means driven by a human being, e.g. hand driven
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61H—PHYSICAL THERAPY APPARATUS, e.g. DEVICES FOR LOCATING OR STIMULATING REFLEX POINTS IN THE BODY; ARTIFICIAL RESPIRATION; MASSAGE; BATHING DEVICES FOR SPECIAL THERAPEUTIC OR HYGIENIC PURPOSES OR SPECIFIC PARTS OF THE BODY
- A61H2201/00—Characteristics of apparatus not provided for in the preceding codes
- A61H2201/50—Control means thereof
- A61H2201/5007—Control means thereof computer controlled
- A61H2201/501—Control means thereof computer controlled connected to external computer devices or networks
- A61H2201/5015—Control means thereof computer controlled connected to external computer devices or networks using specific interfaces or standards, e.g. USB, serial, parallel
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61H—PHYSICAL THERAPY APPARATUS, e.g. DEVICES FOR LOCATING OR STIMULATING REFLEX POINTS IN THE BODY; ARTIFICIAL RESPIRATION; MASSAGE; BATHING DEVICES FOR SPECIAL THERAPEUTIC OR HYGIENIC PURPOSES OR SPECIFIC PARTS OF THE BODY
- A61H2201/00—Characteristics of apparatus not provided for in the preceding codes
- A61H2201/50—Control means thereof
- A61H2201/5023—Interfaces to the user
- A61H2201/5043—Displays
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61H—PHYSICAL THERAPY APPARATUS, e.g. DEVICES FOR LOCATING OR STIMULATING REFLEX POINTS IN THE BODY; ARTIFICIAL RESPIRATION; MASSAGE; BATHING DEVICES FOR SPECIAL THERAPEUTIC OR HYGIENIC PURPOSES OR SPECIFIC PARTS OF THE BODY
- A61H2201/00—Characteristics of apparatus not provided for in the preceding codes
- A61H2201/50—Control means thereof
- A61H2201/5023—Interfaces to the user
- A61H2201/5048—Audio interfaces, e.g. voice or music controlled
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61H—PHYSICAL THERAPY APPARATUS, e.g. DEVICES FOR LOCATING OR STIMULATING REFLEX POINTS IN THE BODY; ARTIFICIAL RESPIRATION; MASSAGE; BATHING DEVICES FOR SPECIAL THERAPEUTIC OR HYGIENIC PURPOSES OR SPECIFIC PARTS OF THE BODY
- A61H2201/00—Characteristics of apparatus not provided for in the preceding codes
- A61H2201/50—Control means thereof
- A61H2201/5058—Sensors or detectors
- A61H2201/5092—Optical sensor
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61H—PHYSICAL THERAPY APPARATUS, e.g. DEVICES FOR LOCATING OR STIMULATING REFLEX POINTS IN THE BODY; ARTIFICIAL RESPIRATION; MASSAGE; BATHING DEVICES FOR SPECIAL THERAPEUTIC OR HYGIENIC PURPOSES OR SPECIFIC PARTS OF THE BODY
- A61H2205/00—Devices for specific parts of the body
- A61H2205/02—Head
- A61H2205/022—Face
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61H—PHYSICAL THERAPY APPARATUS, e.g. DEVICES FOR LOCATING OR STIMULATING REFLEX POINTS IN THE BODY; ARTIFICIAL RESPIRATION; MASSAGE; BATHING DEVICES FOR SPECIAL THERAPEUTIC OR HYGIENIC PURPOSES OR SPECIFIC PARTS OF THE BODY
- A61H2230/00—Measuring physical parameters of the user
- A61H2230/85—Contour of the body
- A61H2230/855—Contour of the body used as a control parameter for the apparatus
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30196—Human being; Person
- G06T2207/30201—Face
Definitions
- the present invention relates to an image processing apparatus, an image processing method, and an image processing program.
- Patent Document 1 it is possible to extract feature points that characterize the degree of fatigue of the user from the face image by the feature extraction unit, and display information on the degree of fatigue calculated from these feature points with a mirror or the like.
- the fixtures with mirrors are disclosed.
- the display unit simply displays the facial image before massage and the facial image after massage side by side.
- the user only confirms the effect of the massage by comparing the two facial images. That is, in the technique described in Patent Document 1, it was impossible to confirm whether the treatment was performed correctly during the beauty treatment and to quantitatively evaluate the effect of the beauty treatment.
- the present invention provides an image processing device, an image processing method, and an image processing program that enable it to confirm whether or not the treatment is performed correctly during the beauty treatment and / or to quantitatively evaluate the effect of the beauty treatment.
- the purpose is to provide.
- the image processing apparatus includes an acquisition means for acquiring an image of an captured object and a detection means for detecting feature points in a real image of the object included in the image.
- a comparison means for acquiring the comparison result by comparing the locus as a tracking result of the movement of the feature point of the real image included in the acquisition target with the preset locus information, and an output for outputting the comparison result. It is characterized by providing means.
- the image processing apparatus the image processing method, and the image processing program of the present invention, it is possible to confirm whether the treatment is performed correctly during the beauty treatment and / or to quantitatively evaluate the effect of the beauty treatment. Become.
- the image processing device 1 is a smart mirror configured as a self-supporting mirror that can be carried by a user.
- the image processing device 1 captures an image of a user as a target person who visually recognizes a mirror.
- the image processing device 1 detects feature points from the user's face based on an image of the user, especially the user's face, and continuously images the user's face to track the movement of the feature points.
- the image processing device 1 compares the locus as the tracking result with the preset locus information to output whether or not the user is correctly performing the treatment during the beauty treatment and the effect of the treatment.
- FIG. 1 is a block diagram showing an overall configuration of an image processing system I including the image processing device 1 according to the present embodiment.
- the image processing system I includes a plurality of image processing devices 1, a network 2, and a server group 3.
- the number of image processing devices 1 is not particularly limited, and n image processing devices 1 (n is an arbitrary natural number) may be included in the image processing system I.
- n image processing devices 1 are described without particular distinction, the alphabet at the end of the reference numeral is omitted, and the term "image processing device 1" is simply referred to.
- the image processing device 1 is a device that detects a user's feature points from a video, tracks the movement of the detected feature points, compares the trajectory as the tracking result with preset trajectory information, and displays the comparison result. Is.
- the image processing device 1 is connected to each server included in the server group 3 via the network 2 so as to be able to communicate with each other.
- the network 2 is realized by, for example, the Internet, a LAN (Local Area Network), a mobile phone network, or a network in which these are combined.
- LAN Local Area Network
- mobile phone network or a network in which these are combined.
- the server group 3 includes various servers that cooperate with the image processing device 1.
- the server group 3 includes an authentication server for authenticating the user of the image processing device 1.
- the server group 3 includes an application distribution server that distributes application software for realizing the function of the image processing device 1.
- the server group 3 includes a measurement data storage server that stores user profile information, which is information including setting information about the user and usage history of the image processing device 1 by the user.
- the image processing system I shown in FIG. 1 is only an example, and servers having other functions may be included in the server group 3. Further, the plurality of servers included in the server group 3 may be realized by separate server devices, or may be realized by a single server device.
- FIG. 2 is a configuration diagram showing an external configuration of the front surface of the image processing device 1 according to the embodiment of the present invention.
- FIG. 3 is a configuration diagram showing an external configuration of a side surface of the image processing device 1.
- the size of the front surface of the image processing apparatus 1 is formed to be, for example, A4 size defined by ISO (International Organization for Standardization) 216, which is an international standard.
- the image processing device 1 includes a main body portion 30, a leg portion 31, and a hinge portion 32.
- the main body portion 30 is a portion including a display unit 18 and other hardware described later with reference to FIG.
- the leg portion 31 and the hinge portion 32 are members for making the image processing device 1 independent.
- the leg portion 31 is rotatably supported by the hinge portion 32 with respect to the main body portion 30.
- the side surface of the main body portion 30 and the side surface of the leg portion 31 can be aligned and carried in a shape that is not bulky.
- the image processing device 1 when the image processing device 1 is installed on a desk or the like and used, the user rotates the leg portion 31 around the hinge portion 32 as a center point.
- the image processing device 1 can be installed independently.
- the hinge portion 32 has a mechanism for holding the leg portion 31 in a state of maintaining a predetermined angle.
- the main body 30 includes the display 18 as described above.
- the display unit 18 is a portion that displays various kinds of information to the user by displaying various kinds of information.
- the display unit 18 is, for example, a user image (corresponding to the user image 51 in the figure) which is a real image of the user captured by the imaging unit 16 as a subject, or an avatar image (avatar in the figure) which is a substitute image for the user.
- a guide image (corresponding to the guide image 53 in the figure), which is auxiliary information for performing guidance, and a biometric information image (corresponding to the biometric information image 54 in the figure) showing the user's biometric information. indicate.
- the display unit 18 synthesizes the guide image with the avatar image and displays the guide image superimposed.
- the user can grasp these various information at once by visually recognizing the display unit 18.
- the display of the display unit 18 has no apparent gap and has a sense of unity suitable for the user to visually recognize.
- the image processing device 1 further includes an imaging unit 16, an input unit 17, and a display unit 18 as an external configuration.
- the image pickup unit 16 is a portion that captures an image of a user facing the display unit 18 as a subject when the image processing device 1 is used.
- the imaging unit 16 is arranged at a position where the user's face image facing the display unit 18 can be imaged. For example, as shown in the drawing, the imaging unit 16 is arranged on the front surface of the main body 30 and above the display unit 18.
- the input unit 17 is a part that receives an operation input by the user.
- the input unit 17 is realized by, for example, a plurality of buttons.
- a button for switching to various modes such as small face esthetics, smile training, and recording of biological information
- a button for switching the power of the image processing device 1 on / off are shown. ..
- the external structure of the image processing device 1 has been explained above. However, this structure is only an example, and the appearance structure of the image processing apparatus 1 is not limited to this example.
- the image processing device 1 may further include a light emitting unit that emits light to illuminate the user facing the display unit 18.
- the image processing device 1 functions as an illuminated mirror when the light emitting unit illuminates the user by adjusting the illuminance and the color component.
- the number of light emitting units may be plural. Further, the light emitting unit may be arranged at the upper part or the lower part of the display unit 18, or may be arranged at the entire periphery of the display unit 18.
- the number and arrangement of the input units 17 may be changed.
- a part of the display unit 18 may be configured as a touch panel, and the input unit 17 and the display unit 18 may be integrally configured.
- FIG. 4 is a block diagram showing a hardware configuration of the image processing device 1.
- the image processing device 1 includes a CPU (Central Processing Unit) 11 which is a processor, a ROM (Read Only Memory) 12, a RAM (Random Access Memory) 13, a bus 14, and an input / output interface. It includes 15, an imaging unit 16, an input unit 17, a display unit 18, a storage unit 19, a communication unit 20, a drive 21, and a battery 22.
- a CPU Central Processing Unit
- ROM Read Only Memory
- RAM Random Access Memory
- the CPU 11 executes various processes according to the program recorded in the ROM 12 or the program loaded from the storage unit 19 into the RAM 13.
- the CPU 11, ROM 12 and RAM 13 are connected to each other via the bus 14.
- An input / output interface 15 is also connected to the bus 14.
- An image pickup unit 16, an input unit 17, a display unit 18, a storage unit 19, a communication unit 20, a drive 21, and a battery 22 are connected to the input / output interface 15.
- the imaging unit 16 includes an optical lens unit and an image sensor.
- the optical lens unit is composed of a lens that collects light, such as a focus lens or a zoom lens, in order to image a subject.
- the focus lens is a lens that forms a subject image on the light receiving surface of the image sensor.
- a zoom lens is a lens that freely changes the focal length within a certain range.
- the imaging unit 16 is also provided with peripheral circuits for adjusting setting parameters such as focus, exposure, and white balance, if necessary.
- the image sensor is composed of a photoelectric conversion element, AFE (Analog Front End), and the like.
- the photoelectric conversion element is composed of, for example, a CMOS (Complementary Metal Oxide Semiconductor) type photoelectric conversion element or the like.
- a subject image is incident on the photoelectric conversion element from the optical lens unit. Therefore, the photoelectric conversion element performs photoelectric conversion (imaging) of the subject image, accumulates an image signal for a certain period of time, and sequentially supplies the accumulated image signal as an analog signal to AFE.
- the AFE executes various signal processing such as A / D (Analog / Digital) conversion processing on the analog image signal.
- a digital signal is generated by various signal processing and is output as an output signal of the imaging unit 16. The output signal of such an imaging unit 16 is appropriately supplied to the CPU 11 and the like.
- the input unit 17 is composed of various buttons, microphones, etc., and inputs various information according to the user's instruction operation and instruction voice.
- the display unit 18 is composed of a liquid crystal display or the like, and displays an image corresponding to the image data output by the CPU 11.
- the storage unit 19 is composed of a semiconductor memory such as a DRAM (Dynamic Random Access Memory) and stores various data.
- a semiconductor memory such as a DRAM (Dynamic Random Access Memory) and stores various data.
- the communication unit 20 controls communication for the CPU 11 to communicate with other devices (for example, each server included in the server group 3) via the network 2.
- the drive 21 is composed of an interface on which the removable media 100 can be mounted.
- a removable media 100 made of a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is appropriately mounted on the drive 21.
- the removable media 100 stores various data such as a program for executing a composite display process described later and image data.
- Various data such as programs and image data read from the removable media 100 by the drive 21 are installed in the storage unit 19 as needed.
- the battery 22 is configured to be rechargeable by supplying electric power to each part and connecting to an external power source.
- the image processing device 1 When the image processing device 1 is not connected to an external power source, the image processing device 1 operates by the electric power of the battery 22.
- the image processing device 1 may further include other hardware in addition to the above-mentioned hardware.
- the image processing device 1 may be composed of a lamp, a speaker, a vibration motor, or the like, and may further include an output unit or the like that outputs light, sound, or a vibration signal.
- FIG. 5 is a functional block diagram showing a functional configuration for executing image processing among the functional configurations of the image processing device 1.
- the image processing is a series of processes in which the image processing device 1 displays an image based on the change of the feature points acquired from the user.
- a guidance information storage unit 191 As shown in FIG. 5, a guidance information storage unit 191, an avatar information storage unit 192, an actual measurement information storage unit 193, and an evaluation result information storage unit 194 are set in one area of the storage unit 19.
- the guidance information storage unit 191 stores various data related to guidance in the display process.
- the guidance information storage unit 191 stores guide image data, text data, sound data, and the like for generating guidance information to be displayed in the display process.
- the guidance information storage unit 191 stores locus information which is information related to the locus to be drawn by the above-mentioned feature points in the skin massage as a beauty treatment.
- the guidance information storage unit 191 stores distance information which is information related to an ideal distance as the distance between the above-mentioned feature points. More specifically, the distance information may be ideal distance information as the distance that changes due to the partial deformation of the face associated with the massage.
- the guidance information storage unit 191 stores timing information relating to the timing at which the movement should change in the massage.
- the guidance information storage unit 191 includes the display order of a plurality of guidance information in a series of guidance, conditions for switching the displayed guidance information to the next guidance information, data for generating various user interfaces, and the like. Is also stored.
- the guidance information storage unit 191 stores the user profile information generated for the display process.
- the guide image may be image data generated in advance, or may be CG (Computer graphics) or the like generated in real time based on an operation.
- the avatar information storage unit 192 stores various data related to the avatar as a substitute for the real image of the user.
- the avatar information storage unit 192 stores avatar image data for generating an avatar image to be displayed in the composite display process, conditions for displaying which avatar image in guidance, and the like.
- the avatar image may be image data generated in advance, or may be CG or the like generated in real time based on an operation.
- the avatar image is, for example, an image of a character that imitates a human or an animal.
- an arbitrary number of avatar images may be prepared, and may be appropriately selected according to the attributes of the user (for example, the gender, age, and preference of the user) and the content of the guidance.
- the avatar image may be a continuous image group for realizing an animation display of a character or the like.
- each data may be stored as one library for each guidance unit so that it can be easily read when realizing a series of guidances.
- the actual measurement information storage unit 193 stores actual measurement information such as the feature points detected by the detection unit 113, the positions of the reference points included in the feature points, and the distance between the feature points calculated by the comparison unit 115. .. In particular, even if the actual measurement information storage unit 193 stores the initial value of the coordinates of the feature points including the reference point and the initial value of the distance between the feature points as distance information before starting the massage as a beauty treatment. Good. More specifically, the distance information may be distance information that changes due to partial deformation of the face associated with the massage.
- the evaluation unit 116 provides information for generating evaluation result information based on the comparison result between the tracking result of the movement of the feature point by the comparison unit 115 and the preset trajectory information. Information for displaying the evaluation result, information indicating the evaluation result, and the like are stored. The evaluation process by the evaluation unit 116 will be described later.
- the information stored in the guidance information storage unit 191, the avatar information storage unit 192, the actual measurement information storage unit 193, and the evaluation result information storage unit 194 as described above may be stored only in the storage unit 19.
- the drive 21 may appropriately store the removable media 100.
- each information stored in the guidance information storage unit 191, the avatar information storage unit 192, the actual measurement information storage unit 193, and the evaluation result information storage unit 194 is appropriately stored in the measurement data storage server or the like included in the server group 3. You may do so.
- the setting processing unit 111 the acquisition unit 112, the detection unit 113, the correction unit 114, the comparison unit 115, the evaluation unit 116, and the display control unit 117 function. ..
- the setting processing unit 111 is a part that controls settings and the like related to image processing and display processing. For example, the setting processing unit 111 acquires application software for performing display processing from the application distribution server included in the server group 3 and operates the application software. Further, the setting processing unit 111 authenticates the user who performs the display processing by communicating with the authentication server included in the server group 3, for example. Further, the setting processing unit 111 updates the user profile information in the display processing by communicating with the measurement data storage server included in the server group 3, for example.
- the setting processing unit 111 displays a menu for performing guidance based on the application software for performing the display processing. For example, based on the guidance, a menu that includes options for selecting the content of the guidance, such as "small face esthetics", “smile massage”, “measurement of biological information", and "makeup". indicate. Further, the setting processing unit 111 accepts the selection of any guidance content from the user who has referred to the menu via the input unit 17 or the like. For example, accept the selection of "small face beauty treatment salon”. As a result, display processing is performed for guidance regarding small face esthetics.
- the small face esthetic means, for example, that the user performs lymphatic massage or the like to reduce the swelling of the face by a massage that drains lymph on the user's own face or the like.
- the acquisition unit 112 acquires an image of the object imaged by the image pickup unit 16.
- the "object” is preferably, but not limited to, a human organ, especially a human face.
- the detection unit 113 detects the feature points in the real image included in the image of the object acquired by the acquisition unit 112.
- the "image of the object” is an image including the object itself and its background, and indicates the imaging range (imaging angle of view) of the imaging unit 16.
- the "real image” is the object itself (or an image area showing the object).
- the "image of the object” is an image including an image of a human face
- the "real image of the object” is preferably a real image of a human face. Is not limited.
- the "feature point” is a point (or area) provided for detecting and recognizing the shape of each organ (eyebrows, eyes, nose, mouth, contour) in the user's facial image.
- FIG. 6 shows an example of the above-mentioned feature points and, among other features, a reference point and a reference line that serve as a reference for detecting the movement of the feature points.
- the feature points are arranged along the contour of the eye, the back of the nose and the wing of the nose, the circumference of the mouth, and the contour of the eyebrows and the cheeks extending from the eyebrows.
- 2 points at the inner corner of the eye E1 and E2 in FIG. 6
- 2 points at the corner of the mouth M1 and M2 in FIG. 6
- 2 points at both ends of the back of the nose N1 and N2 in FIG. 6
- a straight line connecting E1 and E2 (reference line A) and a straight line connecting N1 and N2 (reference line B) are candidates for the reference line.
- the distance between E1 and E2 is used as the above-mentioned distance information. For example, when the distance between E1 and E2 changes due to a beauty treatment such as massage, the changed distance is compared with the initially set distance information.
- FIG. 7 shows an example of a reference point and a reference line used for each treatment.
- the feature points (E3 and E4) corresponding to the two outer corners of the eyes, which are circled in FIG. 7, are used as reference points.
- the feature points (T1 and T2) corresponding to the two corners of the eyebrows circled in FIG. 7 are used as reference points.
- the two feature points (N3 and N4) corresponding to both ends of the ala of nose, which are circled in FIG. 7, are used as reference points.
- the correction unit 114 is a real image captured by the image pickup unit 16 when the user's face (real image) is tilted at a predetermined rotation angle or by the distance relationship between the image pickup unit 16 and the user's face. Enlarge or reduce. More specifically, the positional relationship of the plurality of feature points detected by the detection unit 113 is corrected.
- 8A and 8B are diagrams showing a correction method by the correction unit 114. When correcting the inclination of the horizontal line, as shown in FIG. 8A, the angle between the reference line connecting E1 and E2 and the horizontal straight line is detected, and this angle is 0 °. Rotate the image so that When correcting the inclination of the vertical line, as shown in FIG.
- the angle between the reference line connecting N1 and N2 and the straight line in the vertical direction is detected, and this angle is set to 0 °. Rotate the image so that it becomes.
- the image is enlarged or reduced so that the distance between E1 and E2 becomes a preset reference distance.
- the comparison unit 115 tracks the movement of the feature points detected by continuously imaging the object by the imaging unit 16, and compares the trajectory as the tracking result with the preset trajectory information. Further, the comparison unit 115 may compare the timing of this movement with the preset timing information.
- the comparison unit 115 may obtain the distance between the detected feature points and compare this distance with the preset distance information. For example, the comparison unit 115 may compare the distance between the reference point and the reference line with preset distance information.
- 9 to 12 are diagrams for explaining an example of the distance between the reference point and the reference line and the preset distance information, which the comparison unit 115 compares.
- FIG. 9A shows an example of finger movement during massage.
- the cheek and mouth massage described in FIG. 7 is performed.
- the user first raises his cheeks in the direction of (1) with his fingers on his mouth, and in the direction of (2), spreads his fingers to the left and right so as to spread his cheeks.
- the fingers spread in the left-right direction are returned so as to shrink around the cheeks.
- M1 and M2 are set as reference points for the real image captured by the imaging unit 16 as shown in FIG. 9B. Further, the distances from each of M1 and M2 to the reference line connecting the two outer corners of the eyes are set as VR0 and VL0, respectively. Further, the distances from each of M1 and M2 to the reference line passing through the back of the nose are set as HR0 and HL0, respectively.
- the comparison unit 115 stores the set distance information in the actual measurement information storage unit 193.
- the evaluation unit 116 evaluates the comparison result by the comparison unit 115 and generates the evaluation result. For example, the evaluation unit 116 may generate an evaluation result based on the degree of coincidence between the locus as the tracking result of the movement of the feature point and the preset locus information. Further, the evaluation unit 116 may generate an evaluation result based on the degree of coincidence between the timing of movement of the feature points and the preset timing information. Further, the evaluation unit 116 may generate an evaluation result based on the degree of agreement between the detected distances between the plurality of feature points and the distance information. Further, the evaluation unit 116 stores these evaluation results in the evaluation result information storage unit 194, and may generate further evaluation results based on the history of the evaluation results stored in the evaluation result information storage unit 194. Good.
- the display control unit 117 performs a process of outputting the above comparison result, the evaluation result, the history of the evaluation result, and the like to the display unit 18 as an output means.
- the display control unit 117 may be referred to as an "output unit 117".
- the display control unit 117 displays the user image as the main and displays the composite image obtained by combining the avatar image and the guide image as the sub, and displays the user image as the sub and also displays the avatar image.
- a process of switching between the second display mode in which the composite image combined with the guide image is displayed as the main image is executed.
- the avatar image and the guide image which are composite images, can be displayed in a large size in the center of the screen as the main image, and the user image can be displayed in the lower part of the screen in a smaller size than the main image as a sub image.
- the user image can be displayed in a large size in the center of the screen as the main image
- the composite image, the avatar image and the guide image can be displayed in the lower part of the screen in a smaller size than the main image.
- the display control unit 117 automatically arranges and displays the direction in which the user should face in a series of guidance, the massage method, and biological information in an image or text in an area that does not overlap with the face of the user's mirror image or the face of the avatar image. You may do so.
- the display control unit 117 may output guidance information by another method.
- the display control unit 117 may read the guidance information including the sound data and output the voice or music generated from the read guidance information from the speaker.
- the display control unit 117 may change the light emitting state by the light emitting unit.
- FIG. 13A to 13F are flowcharts illustrating an example of the overall flow of the first image processing executed by the image processing apparatus 1 of FIG. 1 having the functional configuration of FIG.
- step S1 the setting processing unit 111 performs the initial setting. For example, the setting processing unit 111 downloads the personal data set for each user ID from the authentication server included in the server group 3.
- step S2 the setting processing unit 111 performs an activation process for displaying the menu screen on the display unit 18.
- step S3 when the user selects each massage and detects it (S3: YES), the process proceeds to step S4. If the user has not yet selected each massage and no selection has been detected (S3: NO), the process proceeds to step S3.
- step S4 the display control unit 117 causes the display unit 18 to display "face alignment" for aligning the face when the user massages.
- step S5 the detection unit 113 detects the coordinates of the normal facial feature points (including the reference points E1, E2, N1, N2, M1, and M2 in the above example).
- step S6 the comparison unit 115 calculates the distance between the reference point and the reference line (VR0, VL0, HR0, HL0, HE0 in the above example).
- step S7 When the detection of the feature points is completed in step S7 (S7: YES), the process proceeds to step S8. If the feature point detection is not yet completed (S7: NO), the process proceeds to step S5.
- step S8 the comparison unit 115 stores the coordinates of the reference point detected in step S5 and the distance data related to the reference point calculated in step S6 in the actual measurement information storage unit 193.
- step S9 the setting processing unit 111 sets the score value S as the evaluation result, which is counted by the score counter, to 0.
- step S10 the setting processing unit 111 sets the massage repetition count value L counted by the repetition counter to 0.
- step S11 the display control unit 117 displays the guidance of the massage by the CG avatar.
- step S12 the display control unit 117 displays an instruction to start the massage to the user.
- the timing value counted by the timing counter is set to 0.
- step S14 the display control unit 117 displays the massage timing on the display unit 18 and utters a voice guide by the vocalization unit (not shown).
- step S15 when the timing value C counted by the timing counter is any of 0 to 2 (S15: 0 to 2), the process proceeds to step S16.
- the timing value C is 3 (S15: 3), the process proceeds to step S17.
- step S16 the detection unit 113 detects the coordinates of the feature points (in the above example, the reference points E1, E2, N1, N2, M1, M2).
- step S17 1 is added to the repetition count value L counted by the repetition counter. After that, the process proceeds to step S13.
- step S19 the correction unit 114 corrects the inclination of the set of feature points by using the reference point of the face at the normal time. More specifically, the correction unit 114 corrects so that the horizontal line passing through E1 and E2 becomes horizontal and the vertical line passing through N1 and N2 becomes vertical.
- step S20 the comparison unit 115 calculates the distance related to the reference point (VR1, VL1, HR1, HL1 in the above example).
- step S21 when the detection by the detection unit 113 is completed (S21: YES), the process proceeds to step S22. If it has not been completed yet (S21: NO), the process proceeds to step S16.
- step S23 the treatment of (1) in FIG. 9A was performed, and the positions of the feature points VR0, VL0, HR0, and HL0 were VR0> VR1, VL0> VL1, HR0 ⁇ HR1, and HL0 ⁇ HL1, respectively.
- step S24 the treatment of (1) in FIG. 9A was performed, and the positions of the feature points VR0, VL0, HR0, and HL0 were VR0> VR1, VL0> VL1, HR0 ⁇ HR1, and HL0 ⁇ HL1, respectively.
- S23: NO the process proceeds to step S25.
- step S24 the comparison unit 115 calculates the rate of change VR from VR0 to VR1 and the rate of change VL from VL0 to VL1 by the following mathematical formulas (1) and (2).
- VR (1- (VR1 / VR0)) * 100
- VL (1- (VL1 / VL0)) * 100 (2)
- the process proceeds to step S36.
- step S25 when the time required for the treatment of (1) in FIG. 9A exceeds a predetermined time limit (S25: YES), the process proceeds to step S26. If it is within the time limit (S25: NO), the process proceeds to step S23.
- step S26 the evaluation unit 116 subtracts 1 from the score value S which is the evaluation result. After that, the process proceeds to step S14.
- step S27 when the value of C is 1 (S27: 1), the process proceeds to step S28.
- step S28 When the value of C is 2 (S27: 2), the process proceeds to step S32.
- step S28 the operation of (2) in FIG. 9A was performed, and the positions of the feature points VR0, VL0, HR0, and HL0 were VR0 ⁇ VR2, VL0 ⁇ VL2, HR0 ⁇ HR2, and HL0 ⁇ HL2, respectively.
- S28: YES the process proceeds to step S29.
- S28: NO the process proceeds to step S30.
- step S29 the comparison unit 115 calculates the rate of change HR from HR0 to HR2 and the rate of change HL from HL0 to HL2 by the following mathematical formulas (3) and (4).
- HR (1- (HR2 / HR0)) * 100
- HL (1- (HL2 / HL0)) * 100 (4)
- the process proceeds to step S36.
- step S30 when the time required for the treatment of (2) in FIG. 9A exceeds a predetermined time limit (S30: YES), the process proceeds to step S31. If it is within the time limit (S30: NO), the process proceeds to step S28.
- step S31 the evaluation unit 116 subtracts 1 from the score value S which is the evaluation result. After that, the process proceeds to step S14.
- step S33 the comparison unit 115 calculates the rate of change HR from HR3 to HR0 and the rate of change HL from HL3 to HL0 by the following mathematical formulas (5) and (6).
- HR (1- (HR0 / HR3)) * 100
- HL (1- (HL0 / HL3)) * 100 (4)
- the process proceeds to step S36.
- step S34 when the time required for the treatment of (3) in FIG. 9A exceeds a predetermined time limit (S34: YES), the process proceeds to step S35. If it is within the time limit (S34: NO), the process proceeds to step S32.
- step S35 the evaluation unit 116 subtracts 1 from the score value S, which is the evaluation result. After that, the process proceeds to step S14.
- step S36 when VR> 10% and VL> 10% (S36: YES), the process proceeds to step S37. In other cases (S36: NO), the process proceeds to step S38.
- step S37 the evaluation unit 116 adds 10 to the score value S, which is the evaluation result. After that, the process proceeds to step S41.
- step S38 when VR> 5% and VL> 5% (S38: YES), the process proceeds to step S39. In other cases (S38: NO), the process proceeds to step S40.
- step S39 the evaluation unit 116 adds 5 to the score value S, which is the evaluation result. After that, the process proceeds to step S41.
- step S40 the evaluation unit 116 adds 1 to the score value S, which is the evaluation result. After that, the process proceeds to step S41.
- step S41 the display control unit 117 displays the score value S, and the vocalization unit (not shown) utters the score value S.
- step S42 the setting processing unit 111 adds 1 to the timing value C counted by the timing counter.
- step S43 the comparison unit 115 stores the data of the coordinates of the reference point and the measured value of the distance related to the reference point in the actual measurement information storage unit 193.
- step S44 when the repetition rate value L counted by the repetition counter exceeds 3, the process proceeds to step S45.
- the repetition count value L does not exceed 3 (S44: NO)
- the process proceeds to step S14.
- step S45 When the massage is completed in step S45 (S45: YES), the process proceeds to step S46. In other cases (S45: NO), the process proceeds to step S3.
- step S46 the image processing device 1 turns off the power.
- FIG. 14A to 14B are flowcharts illustrating an example of the overall flow of the second image processing executed by the image processing apparatus 1 of FIG. 1 having the functional configuration of FIG.
- the second image processing is an image processing executed when the user massages for a long period of time.
- step S51 the setting processing unit 111 inquires of the server group 3 to authenticate the user personally.
- step S52 If the authentication is OK in step S52 (S52: YES), the process proceeds to step S53. If the authentication is NG (S52: NO), the process proceeds to step S51.
- step S53 the setting processing unit 111 reads from the server group 3 the target number of days that the user targets as the number of consecutive massage days.
- step S54 the setting processing unit 111 performs the massage for the number of consecutive days based on the flag attached to the history of the actual measurement data from the server group 3 and indicating whether or not the massage has been performed. Read a certain number of consecutive days.
- step S55: YES If the number of consecutive days is less than the target number of days in step S55 (S55: YES), the process proceeds to step S56. When the number of consecutive days is equal to or greater than the target number of days (S55: NO), the process proceeds to step S57.
- step S56 the display control unit 117 displays on the display unit 18 the number of days until the target date, that is, the number of days obtained by subtracting the number of consecutive days from the target number of days.
- step S57 the setting processing unit 111 adds the target number of days. For example, the setting processing unit 111 adds 28 days for 4 weeks to the current target number of days.
- step S58 If it is detected in step S58 that the treatment is to be started (S58: YES), the process proceeds to step S59. When it is detected that the treatment has not been started (S58: NO), the process proceeds to step S58.
- step S59 the image processing device 1 executes the processes of steps S1 to S46 described in FIGS. 13A to 13F as the beauty treatment determination process.
- step S60 If the end of the treatment is detected in step S60 (S60: YES), the process proceeds to step S61. If the end of the treatment is not detected (S60: NO), the process proceeds to step S59.
- step S61 the evaluation unit 116 stores the score value S after the treatment is completed in the evaluation result information storage unit 194.
- step S62 if the current score value S is equal to or higher than a certain level (S62: YES), the process proceeds to step S63. If the score value S this time is less than a certain value (S62: NO), the process proceeds to step S64.
- step S63 the setting processing unit 111 saves a flag indicating that the treatment with a score value S equal to or higher than a certain level has been performed in the server group 3.
- step S64 the setting processing unit 111 calculates the number of consecutive execution days.
- step S65 If the number of consecutive days exceeds the target number of days in step S65 (S65: YES), the process proceeds to step S66. If the number of consecutive days is less than the target number of days (S65: NO), the process proceeds to step S67.
- step S66 the display control unit 117 displays on the display unit 18 that the target has been cleared.
- step S67 If the end of the treatment is detected in step S67 (S67: YES), the process proceeds to step S68. If the end of the treatment is not detected (S67: NO), the process proceeds to step S58.
- step S68 the setting processing unit 111 saves the target number of days in the server group 3.
- the image processing device 1 of the present embodiment described above includes an acquisition unit 112, a detection unit 113, a comparison unit 115, and an output unit (display control unit) 117.
- the acquisition unit 112 acquires an image of the captured object.
- the detection unit 113 detects the feature points in the real image of the object included in the captured image of the object.
- the comparison unit 115 acquires the comparison result by comparing the locus as the tracking result of the movement of the feature point of the real image included in the acquisition target by the acquisition unit 112 with the preset locus information.
- the output unit 117 outputs the comparison result.
- the detection unit 113 detects a plurality of feature points
- the comparison unit 115 obtains the distance between the detected plurality of feature points and compares the distance with the preset distance information. As a result, it is possible to acquire how much the distance between the feature points has changed compared to the initial setting value, and based on this acquisition result, it is determined whether or not the treatment has been performed correctly on the object. Can be done.
- the imaging is performed during the treatment on the object, and the distance information is the distance information that changes due to the partial deformation of the object due to the treatment.
- the distance information is the distance information that changes due to the partial deformation of the object due to the treatment.
- the image processing device 1 of the present embodiment further includes an evaluation unit 116.
- the evaluation unit 116 evaluates the comparison result by the comparison unit 115 and generates the evaluation result.
- the output unit 117 further outputs the evaluation result. This makes it possible to score how correctly the treatment was performed based on the change in each feature point.
- the evaluation unit 116 evaluates the degree of agreement between the distance and the distance information. Thereby, it is possible to score how correctly the treatment is performed based on how much the distance between each feature point matches the preset distance information.
- the image processing device 1 of the present embodiment further includes an evaluation result information storage unit 194.
- the evaluation result information storage unit 194 stores the history of the evaluation results.
- the output unit 117 outputs the history of the evaluation results stored in the evaluation result information storage unit 194.
- the user of the image processing apparatus 1 can know the history of the evaluation result, that is, the history of the score obtained by scoring how correctly the treatment has been performed so far.
- the output unit 117 outputs that the number of consecutive days is equal to or greater than the target number of days. ..
- the user of the image processing apparatus 1 is motivated to continue the treatment that gives an evaluation result of a certain level or higher.
- the comparison unit 115 compares the movement timing with the preset timing information. As a result, the user of the image processing device 1 can know whether or not the treatment is performed at an appropriate timing.
- the object is a human organ. This makes it possible to determine whether or not a massage or other treatment has been performed correctly on a human organ.
- the object is a human face. This makes it possible to determine whether or not a massage or other treatment has been performed correctly on the human face.
- the rate of change of the distance between the feature points is compared with a predetermined value, and based on the result of the comparison, the degree of correctness of the treatment is scored, but it is limited to this format. is not it.
- the length of the change in the distance between the feature points may be compared with a predetermined value, and the degree of correctness of the treatment may be scored based on the result of the comparison.
- the display unit 18 of the image processing device 1 of the above embodiment with a mirror unit having a reflecting surface.
- the mirror unit is realized by a half mirror having both transmission characteristics and reflection characteristics as optical characteristics.
- the mirror unit is arranged so as to be superimposed in the direction in which the user in front of the display unit 18 can see.
- the user is not the user image captured by the imaging unit 16, but his / her face reflected by the mirror unit and various information displayed on the display unit 18 and transmitted through the mirror unit. (For example, a composite image) can be visually recognized at the same time.
- the user image captured by the imaging unit 16 as the subject is visually recognized as the user's real image, but in this modification, the user's mirror image reflected by the mirror unit is visually recognized as the user's real image. I will let you. Even in this way, the same effect as that of the above-described embodiment can be obtained.
- the image processing device 1 to which the present invention is applied has been described as an example of an electronic device incorporated in a portable self-supporting mirror, but the present invention is not particularly limited thereto.
- the present invention can be applied to an electronic device incorporated in a large mirror such as a full-length mirror, an electronic device incorporated in a stationary vanity, and a mirror-shaped electronic device installed in a bathroom.
- the above-mentioned series of processes can be executed by hardware or software.
- the functional configuration of FIG. 5 is merely an example and is not particularly limited. That is, it is sufficient that the image processing device 1 is provided with a function capable of executing the above-mentioned series of processes as a whole, and what kind of functional block is used to realize this function is not particularly limited to the example of FIG. ..
- one functional block may be configured by a single piece of hardware, a single piece of software, or a combination thereof.
- the functional configuration in the present embodiment is realized by a processor that executes arithmetic processing, and the processor that can be used in the present embodiment is composed of various processing units such as a single processor, a multiprocessor, and a multicore processor. In addition to the above, it includes a combination of these various processing units and processing circuits such as an ASIC (Application Specific Integrated Circuit) and an FPGA (Field-Programmable Gate Array).
- ASIC Application Specific Integrated Circuit
- FPGA Field-Programmable Gate Array
- the computer may be a computer embedded in dedicated hardware. Further, the computer may be a computer capable of executing various functions by installing various programs, for example, a general-purpose personal computer.
- the recording medium including such a program is not only composed of the removable media 100 of FIG. 4 distributed separately from the device main body in order to provide the program to the user, but also is preliminarily incorporated in the device main body. Consists of recording media and the like provided in.
- the removable media 100 is composed of, for example, a magnetic disk (including a floppy disk), an optical disk, a magneto-optical disk, or the like.
- the optical disk is composed of, for example, a CD-ROM (Compact Disc-Read Only Memory), a DVD, a Blu-ray (registered trademark) Disc (Blu-ray disc), or the like.
- the magneto-optical disk is composed of an MD (Mini-Disc) or the like.
- the recording medium provided to the user in a state of being preliminarily incorporated in the apparatus main body is composed of, for example, the ROM 12 of FIG. 4 in which the program is recorded, the hard disk included in the storage unit 19 of FIGS. To.
- the steps for describing a program to be recorded on a recording medium are not only processed in chronological order but also in parallel or individually, even if they are not necessarily processed in chronological order. It also includes the processing to be executed.
- the term of the system means an overall device composed of a plurality of devices, a plurality of means, and the like.
- the present invention is particularly useful for confirming whether the treatment is performed correctly during the beauty treatment and / or for quantitatively evaluating the effect of the beauty treatment.
- Measuring device 2 Network 3 Server group 11 CPU 12 ROM 13 RAM 14 Bus 15 Input / output interface 16 Imaging unit 17 Input unit 18 Display unit 19 Storage unit 20 Communication unit 21 Drive 30 Main unit 31 Legs 32 Hinge 100 Removable media 111 Setting processing unit 112 Acquisition unit 113 Detection unit 114 Correction unit 115 Comparison Unit 116 Evaluation Unit 117 Display Control Unit S Image Processing System
Landscapes
- Health & Medical Sciences (AREA)
- Engineering & Computer Science (AREA)
- Life Sciences & Earth Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Physics & Mathematics (AREA)
- Medical Informatics (AREA)
- Public Health (AREA)
- Biomedical Technology (AREA)
- Veterinary Medicine (AREA)
- Animal Behavior & Ethology (AREA)
- Heart & Thoracic Surgery (AREA)
- Pathology (AREA)
- Molecular Biology (AREA)
- Surgery (AREA)
- Biophysics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Epidemiology (AREA)
- Oral & Maxillofacial Surgery (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Radiology & Medical Imaging (AREA)
- Primary Health Care (AREA)
- Multimedia (AREA)
- Dentistry (AREA)
- Quality & Reliability (AREA)
- Developmental Disabilities (AREA)
- Pain & Pain Management (AREA)
- Business, Economics & Management (AREA)
- General Business, Economics & Management (AREA)
- Child & Adolescent Psychology (AREA)
- Rehabilitation Therapy (AREA)
- Physical Education & Sports Medicine (AREA)
- Hospice & Palliative Care (AREA)
- Dermatology (AREA)
- Social Psychology (AREA)
- Psychology (AREA)
- Psychiatry (AREA)
- Human Computer Interaction (AREA)
- Image Processing (AREA)
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202080066138.5A CN114430663B (zh) | 2019-09-24 | 2020-09-23 | 图像处理装置、图像处理方法以及存储介质 |
| US17/762,258 US12315164B2 (en) | 2019-09-24 | 2020-09-23 | Image processing device, image processing method, and storage medium |
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| JP2019-173461 | 2019-09-24 | ||
| JP2019173461A JP7419720B2 (ja) | 2019-09-24 | 2019-09-24 | 画像処理装置、画像処理方法及び画像処理プログラム |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| WO2021060250A1 true WO2021060250A1 (ja) | 2021-04-01 |
Family
ID=75158225
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| PCT/JP2020/035722 Ceased WO2021060250A1 (ja) | 2019-09-24 | 2020-09-23 | 画像処理装置、画像処理方法及び画像処理プログラム |
Country Status (4)
| Country | Link |
|---|---|
| US (1) | US12315164B2 (cg-RX-API-DMAC7.html) |
| JP (1) | JP7419720B2 (cg-RX-API-DMAC7.html) |
| CN (1) | CN114430663B (cg-RX-API-DMAC7.html) |
| WO (1) | WO2021060250A1 (cg-RX-API-DMAC7.html) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20220100410A (ko) * | 2021-01-08 | 2022-07-15 | 삼성전자주식회사 | 전자 장치 및 그의 아바타 생성을 위한 정보를 관리하는 방법 |
| KR102988615B1 (ko) * | 2023-02-22 | 2026-07-09 | 코스맥스 주식회사 | 시술 효과를 강조하는 안면 데이터 관리 장치 및 방법 |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2005242567A (ja) * | 2004-02-25 | 2005-09-08 | Canon Inc | 動作評価装置及び方法 |
| JP2016101365A (ja) * | 2014-11-28 | 2016-06-02 | パナソニックIpマネジメント株式会社 | 皺ケア支援装置および皺ケア支援方法 |
Family Cites Families (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN100565583C (zh) * | 2004-11-12 | 2009-12-02 | 欧姆龙株式会社 | 人脸特征点检测装置、特征点检测装置 |
| JP2009213751A (ja) * | 2008-03-12 | 2009-09-24 | Sony Ericsson Mobilecommunications Japan Inc | 化粧評価プログラム、化粧評価方法および化粧評価装置 |
| WO2010086993A1 (ja) * | 2009-01-30 | 2010-08-05 | 富士通フロンテック株式会社 | 認証装置、撮像装置、認証方法および認証プログラム |
| JPWO2012096081A1 (ja) * | 2011-01-15 | 2014-06-09 | 株式会社 資生堂 | マッサージの評価方法及び装置、並びにプログラム及びコンピュータ読み取り可能な記憶媒体 |
| JP2012152389A (ja) | 2011-01-26 | 2012-08-16 | Panasonic Corp | ミラー付什器 |
| AU2013203345A1 (en) * | 2012-01-11 | 2013-07-25 | Steven Liew | A method and apparatus for facial aging assessment and treatment management |
| AU2015258807B2 (en) * | 2014-05-16 | 2018-11-29 | Corcept Therapeutics, Inc. | Systems and methods of managing treatment of a chronic condition by symptom tracking |
| TWI669103B (zh) * | 2014-11-14 | 2019-08-21 | 日商新力股份有限公司 | 資訊處理裝置、資訊處理方法及程式 |
| TWI585711B (zh) * | 2016-05-24 | 2017-06-01 | 泰金寶電通股份有限公司 | 獲得保養信息的方法、分享保養信息的方法及其電子裝置 |
| CN106940792B (zh) * | 2017-03-15 | 2020-06-23 | 中南林业科技大学 | 基于特征点运动的人脸表情序列截取方法 |
| DE102017212176A1 (de) * | 2017-07-17 | 2019-01-17 | Henkel Ag & Co. Kgaa | System und Verfahren zum Ermitteln einer Wirksamkeit einer kosmetischen Hautbehandlung |
| CN108764071B (zh) * | 2018-05-11 | 2021-11-12 | 四川大学 | 一种基于红外和可见光图像的真实人脸检测方法及装置 |
-
2019
- 2019-09-24 JP JP2019173461A patent/JP7419720B2/ja active Active
-
2020
- 2020-09-23 US US17/762,258 patent/US12315164B2/en active Active
- 2020-09-23 CN CN202080066138.5A patent/CN114430663B/zh active Active
- 2020-09-23 WO PCT/JP2020/035722 patent/WO2021060250A1/ja not_active Ceased
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2005242567A (ja) * | 2004-02-25 | 2005-09-08 | Canon Inc | 動作評価装置及び方法 |
| JP2016101365A (ja) * | 2014-11-28 | 2016-06-02 | パナソニックIpマネジメント株式会社 | 皺ケア支援装置および皺ケア支援方法 |
Also Published As
| Publication number | Publication date |
|---|---|
| US12315164B2 (en) | 2025-05-27 |
| CN114430663B (zh) | 2024-10-15 |
| JP7419720B2 (ja) | 2024-01-23 |
| CN114430663A (zh) | 2022-05-03 |
| US20220343506A1 (en) | 2022-10-27 |
| JP2021051493A (ja) | 2021-04-01 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP7135472B2 (ja) | 表示制御装置、表示制御方法及び表示制御プログラム | |
| US10863812B2 (en) | Makeup compact with eye tracking for guidance of makeup application | |
| KR20180136387A (ko) | 커뮤니케이션 장치, 커뮤니케이션 로봇 및 컴퓨터가 판독 가능한 기록 매체 | |
| CN110993051B (zh) | 判定装置、判定方法以及记录介质 | |
| JP2013243648A (ja) | 写真シール機、写真シール機の処理方法、並びにプログラム | |
| CN112741609B (zh) | 电子装置、电子装置的控制方法以及介质 | |
| EP2055361A1 (en) | Voice outputting method and device linked to images | |
| JP7419720B2 (ja) | 画像処理装置、画像処理方法及び画像処理プログラム | |
| CN110297720B (zh) | 通知装置、通知方法及存储通知程序的介质 | |
| JP2020057153A (ja) | 表示制御装置、表示制御方法及び表示制御プログラム | |
| JP2022052451A (ja) | 電子装置、電子装置の制御方法及び電子装置の制御プログラム | |
| JP7392806B2 (ja) | 電子装置、電子装置の制御方法及び電子装置の制御プログラム | |
| US11800989B2 (en) | Electronic device, control method for the electronic device, and storage medium | |
| WO2015152028A1 (ja) | 化粧支援装置および記録媒体 | |
| JP7135466B2 (ja) | 表示装置、表示方法及び表示プログラム | |
| US10915617B2 (en) | Image processing apparatus, image processing method, and recording medium | |
| JP7415318B2 (ja) | 画像処理装置、画像処理方法及びプログラム | |
| JP2019208906A (ja) | 情報処理装置、情報処理方法及び情報処理プログラム | |
| US7679580B2 (en) | Time-lapsing mirror |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| 121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 20870223 Country of ref document: EP Kind code of ref document: A1 |
|
| NENP | Non-entry into the national phase |
Ref country code: DE |
|
| 122 | Ep: pct application non-entry in european phase |
Ref document number: 20870223 Country of ref document: EP Kind code of ref document: A1 |
|
| WWG | Wipo information: grant in national office |
Ref document number: 17762258 Country of ref document: US |