KR102216892B1 - Psychology consultation method capable of personalized video consultation through real-time image mapping - Google Patents

Abstract

A psychology consultation method capable of personalized video consultation through real-time video mapping includes the steps of: receiving a counselor video from an imaging unit; receiving a covering image from a counselee terminal; creating a mapping video by mapping the covering video to the counselor video; transmitting the mapping image to the counselee terminal; receiving response information of a counselee from a response information measuring unit while the mapping video is reproduced in the counselee terminal; creating consultation result information based on the response information; and transmitting counseling result information to the counselee terminal. Therefore, the counselee can participate in counseling with a more comfortable mind.

Description

Psychological counseling method that enables personalized video counseling through real-time image mapping {PSYCHOLOGY CONSULTATION METHOD CAPABLE OF PERSONALIZED VIDEO CONSULTATION THROUGH REAL-TIME IMAGE MAPPING}

The present invention relates to a psychological counseling method, and more particularly, to a psychological counseling method capable of personalized video counseling through real-time image mapping.

The number of counselees visiting the Department of Mental Health Medicine is increasing day by day. Although the prejudice has not completely disappeared yet, in order to recover a healthy mind in a busy daily life, even though he has not been diagnosed with a specific disease, he visits regularly to eliminate the rejection of counseling and is paying more and more attention to the health of the mind.

In general, in mental health medical consultation, experts such as doctors or clinical psychologists determine the current condition through consultation with the counselee, and treatment and prescription are performed based on this.

The counselee proceeds with several tests provided by the counselor and conducts counseling based on them to share the current state. For visitors who have previously visited or who clearly understand their condition, smooth consultation with the counselor and follow-up treatment are generally possible. However, when visiting for the first time or when the current condition or cause is not yet identified, smooth counseling and treatment are difficult only with existing tests and a few questions based on this.

During the initial visit, it is possible to reduce the burden on the counselor and conduct counseling in a more comfortable environment, so that the counselor and the counselor can smoothly consult and share information about the current state more clearly and positively, and both identify the cause of the cause and have a healthy mind. We need a means to find a way to find it.

Korean Patent Publication No. 10-2018-0099403

The present invention is to solve the above-described problems, by providing a personalized image in which a covering image desired by the counselee is mapped in real time to an image photographed by the counselor, the counselee can participate in the counseling more comfortably and at the same time, EEG, Its purpose is to provide a psychological counseling method capable of grasping the factors and changing patterns of the psychological state by collecting and analyzing the reaction information of the subject such as pulse and motion.

The subject of the present invention is not limited to the problems mentioned above, and other problems that are not mentioned will be clearly understood by those skilled in the art from the following description.

A psychological counseling method capable of personalized video counseling through real-time image mapping according to an embodiment of the present invention for solving the above problems includes receiving a counselor image from an image photographing unit, and receiving a covering image from a target terminal. And, generating a mapping image by mapping the covering image to the counselor image, transmitting the mapping image to the counselee terminal, and receiving response information of the counselor from the response information measuring unit while the mapping image is played back in the counselor terminal. And generating counseling result information based on response information, and transmitting counseling result information to the counselee terminal.

According to an embodiment of the present invention, the generating of the mapping image may include extracting 3D facial coordinate information of a cover person appearing in the covering image; And mapping the cover person to the counselor appearing in the counselor image based on the 3D facial coordinate information of the cover person appearing in the covering image, wherein the generating of the counseling result information includes the response information Generating analysis information on the basis of; Generating new consultation result information by matching image information on the mapping image with the analysis information; Generating the consultation result information including the existing consultation result information and the new consultation result information; And adding the new counseling result information to the existing counseling result information, and the response information may include at least one of pulse information, brainwave information, and motion information of the counselee.

According to an embodiment of the present invention, the analysis information is. It may include abnormal pulse information and abnormal EEG information measured as values outside a predetermined range, and motion information during a time when the abnormal pulse information or the abnormal EEG information is generated.

Advantageous Effects of Invention According to the present invention, it is possible to indirectly confront a person who is difficult to face in reality and obtain counseling effects such as recognizing the current mental state, recognizing the change in the mental state, and improving the mental state. The counselee can recognize the change in the psychological state and the mental state, and it is possible to grasp the change in the psychological state by grasping the change compared to the previous psychological counseling.

In other words, the counselee can simultaneously check his/her body change while indirectly contacting the cover person in the covering image that he/she wants to face through the mapping image.Therefore, the type of psychological state that is difficult to recognize, the change mode of the mental state, internal and external Factors, etc. can be recognized, and psychological changes can be identified in repeated psychological counseling, and a direction for improving the mental state can be sought together.

However, the effects of the present invention are not limited to the effects mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the following description.

1 is a block diagram of a psychological counseling system capable of personalized video counseling through real-time image mapping according to an embodiment of the present invention.
2 is an exemplary configuration diagram of a response information measuring unit of the psychological counseling system of FIG. 1.
3 is an exemplary configuration diagram of a counseling information processing unit of the psychological counseling system of FIG. 1.
4 is a flowchart illustrating steps of a psychological counseling method capable of personalized video counseling through real-time image mapping according to an embodiment of the present invention.
5 is a flowchart illustrating each step of a step of generating counseling result information in the psychological counseling method of FIG. 4.
6 exemplarily shows the internal configuration of a computing device that executes a psychological counseling method capable of personalized video counseling through real-time image mapping according to an embodiment of the present invention.

Advantages and features of the present invention, and a method of achieving them will become apparent with reference to the embodiments described below in detail together with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but will be implemented in various forms different from each other, and only these embodiments make the disclosure of the present invention complete, and common knowledge in the technical field to which the present invention pertains. It is provided to completely inform the scope of the invention to those who have it, and the invention is only defined by the scope of the claims.

Various aspects and features will be presented by a system that may include multiple devices, components and/or modules, and the like. It is also noted that various systems may include additional devices, components and/or modules, and/or may not include all of the devices, components, modules, etc. discussed in connection with the figures. It must be understood and recognized.

As used herein, “an embodiment,” “example,” “aspect,” “example,” and the like may not be construed as having any aspect or design described as being better or advantageous than other aspects or designs. . The terms'~part','component','module','system', and'interface' used below generally mean a computer-related entity, for example, hardware, hardware It can mean a combination of software and software, or software.

In the embodiments of the present invention, unless otherwise defined, all terms used herein, including technical or scientific terms, are the same as those generally understood by those of ordinary skill in the art to which the present invention belongs. It has meaning. Terms as defined in a commonly used dictionary should be interpreted as having a meaning consistent with the meaning in the context of the related technology, and unless explicitly defined in the embodiments of the present invention, an ideal or excessively formal meaning Is not interpreted as.

In describing the present invention, if it is determined that a detailed description of related known technologies may unnecessarily obscure the subject matter of the present invention, a detailed description thereof will be omitted. When'include','have','consists of' and the like mentioned in the specification are used, other parts may be added unless'only' is used. In the case of expressing the constituent elements in the singular, it includes the case of including the plural unless specifically stated otherwise.

Each of the features of the various embodiments of the present invention can be partially or entirely combined or combined with each other, and as a person skilled in the art can fully understand, technically various interlocking and driving are possible, and each of the embodiments may be independently implemented with respect to each other. It may be possible to do it together in a related relationship.

Hereinafter, a psychological counseling method capable of personalized video counseling through real-time image mapping according to the present invention will be described with reference to the accompanying drawings.

Referring to FIG. 1, a psychological counseling system capable of personalized video counseling through real-time image mapping includes a response information measuring unit 1000, an image capturing unit 2000, a counseling information processing unit 3000, and a target terminal 4000. , And a counselor terminal 5000.

The reaction information measuring unit 1000 collects reaction information generated by the reaction of the subject. Reaction information is biometric information such as EEG, pulse, body temperature, blood sugar, pupil changes, blood pressure, dissolved oxygen in blood, etc. that can be measured from the subject's body through a measurement module attached or worn on the subject's body, and detects the subject's movement. It includes motion information that can be measured through a capable detection module. The response information measurement unit 1000 transmits the collected response information to the consultation information processing unit 3000.

The image photographing unit 2000 photographs including the face of the counselor, and transmits the photographed image to the counseling information processing unit 3000. The image capturing unit 2000 may include various types of cameras capable of capturing images through an image sensor, and it is good in terms of ease of mapping to include an RGBD camera capable of acquiring 3D facial coordinate information of a consultant.

The counseling information processing unit 3000 generates, processes, transmits and receives, and stores counseling information, and transmits the counseling information to the counselee terminal 4000 and the counselor terminal 5000. The counseling information includes the above-described response information, image information related to an image to be provided to the counselee, an image to be provided to the counselor, response information, image information, and result information derived based on the image. The consultation information processing unit 3000 receives response information from the response information measurement unit 1000. The consultation information processing unit 3000 receives an image photographed from the image photographing unit 2000. The counseling information processing unit 3000 induces active counseling participation by the counselee by providing counseling information to the counselee, and can grasp a change factor and change pattern of the psychological state of the counselee.

The counselee terminal 4000 provides the counseling information received from the counseling information processing unit 3000 in a form that the counselee can check. The counselee terminal 4000 may provide image information, video, and counseling result information to the counselee to induce a change in psychology of the counselee, an action according to an intention, and an understanding of the counseling result.

The counselor terminal 5000 provides the counseling result information received from the counseling information processing unit 3000 in a form that the counselor can check. The counselor terminal 5000 provides information including response information, video information, video, counseling result information, intermediate calculation information derived from the process of processing the counseling information by the counseling information processor 3000 to the counselor. It can induce the counselor's psychological change, the counselor's understanding of intention, and the recognition of the counseling result.

The consultant terminal 4000 and the consultant terminal 5000 may be communication devices capable of network access such as a desktop computer, a laptop computer, a smart phone, a personal digital assistant (PDA), a portable multimedia player (PMP), a tablet PC, etc. There is no limit to the number of me. In particular, the target terminal 4000 includes a head mounted display (HMD) device for realizing virtual reality, and may enhance the reality of an indirect experience in the process of conducting psychological counseling while watching a mapping image to be described later.

Referring to FIG. 2, the response information measurement unit 1000 includes a pulse rate measurement module 1100, an EEG measurement module 1200, and a motion detection module 1300. The response information measuring unit 1000 transmits the collected response information of the counselee to the counseling information processing unit 3000.

The pulse measurement module 1100 is worn on the wrist of the subject to measure the pulse rate of the subject. Pulse information that can be obtained through pulse measurement includes heart rate, heart rate variability index, sympathetic activity index, parasympathetic activity index, autonomic activity index, and autonomic balance index. Pulse information of the counselee measured by the pulse rate measurement module 1100 may be transmitted to the counseling information processing unit 3000 and used to determine the psychological state of the counselee.

The pulse measurement module 1100 may include a sensor for measuring pulse rate, such as a photoplethysmography (PPG) sensor and an electrorocardiogram (ECG) sensor. In addition, the pulse rate measurement module 1100 may be provided with known elements and circuits required to process a signal generated from a sensor and transmit it to the consultation information processing unit 3000.

The EEG measurement module 1200 is attached to the head of the subject to measure the EEG of the subject. EEG information obtained through EEG measurement includes information on one or more frequency domains of delta, theta, alpha, beta, and gamma measured from the left and right brains. The EEG information of the counselee measured by the EEG measurement module 1200 may be transmitted to the counseling information processing unit 3000 and used to determine the psychological state of the counselee.

The EEG measurement module 1200 may include a sensor for measuring EEG, such as an Electroencephalogram (EEG) sensor. In addition, the EEG measurement module 1200 may be provided with known elements and circuits necessary for processing a signal generated from a sensor and transmitting it to the consultation information processing unit 3000.

The motion detection module 1300 is worn, attached, or held by the counselor to detect the motion of the counselor. Motion information that can be obtained through motion detection includes input motions such as selecting a part of two or more options or executing a function according to video playback, an expression motion required for consultation through video, and counseling. It may include psychological movements to identify psychological changes such as concentration, tension, and embarrassment of the counselee. The motion information of the counselee measured by the motion detection module 1300 may be transmitted to the counseling information processing unit 3000 and used to grasp the counselee's intentions, requirements, and mental state.

The motion detection module 1300 may include sensors for detecting various motions, such as a gyro sensor, an acceleration sensor, a pressure sensor, a torque sensor, and an iris recognition sensor. In addition, the motion detection module 1300 may include known elements and circuits required to process a signal generated from a sensor and transmit it to the consultation information processing unit 3000.

Referring to FIG. 3, the consultation information processing unit 3000 may include a processor 3100, a memory 3200, a network interface 3300, and a bus (shown by an arrow). The memory 3200 may include an operating system (OS) 3210, a program execution code 3220, and information on a consultant 3230. The consultation information processing unit 3000 may be a personal computer or a server. For example, the consultation information processing unit 3000 includes mobile devices such as smartphones and tablets, PCs such as desktops and laptops, mobile medical devices, digital cameras, and wearables. It may be a device or a smart watch.

The processor 3100 may include an image processing unit 3110, a response information collection unit 3130, a response information analysis unit 3140, and a consultation information providing unit 3150. In another embodiment, the consultation information processing unit 3000 may further include components different from the components shown in FIG. 3.

The processor 3100 may be configured to process instructions of a computer program by performing input/output operations such as basic arithmetic, logic, and an image optimization method for a compressed package file. Instructions may be provided to the processor 3100 through the memory 3200 or the network interface 3300 or a bus. The processor 3100 may be configured to execute program codes for the image processing unit 3110, the reaction information collection unit 3130, the reaction information analysis unit 3140, and the consultation information providing unit 3150. The program code may be stored in a recording device such as the memory 3200.

The processor 3100 is to implement a consultation information processing unit 3000 including an image processing unit 3110, a reaction information collection unit 3130, a reaction information analysis unit 3140, and a consultation information providing unit 3150. Can be configured. The processor 3100 may be configured such that some components are omitted, additional components not shown are further included, or two or more components are combined according to a method of implementing the counseling information processing unit 3000.

The memory 3200 is a computer-readable recording medium and may include a permanent mass storage device such as a random access memory (RAM), read only memory (ROM), and a disk drive. In the memory, the OS 3210, the program execution code 3220, and the counselor information 3230 including response information, video information, counseling result information, and personal information of the counselee may be stored. These software components may be loaded from a separate computer-readable recording medium such as a floppy drive, disk, tape, DVD/CD-ROM drive, or memory card. In another embodiment, software components may be loaded into the memory 3200 through the network interface 3300 rather than a computer-readable recording medium.

The network interface 3300 may be a computer hardware component for connecting the consultation information processing unit 3000 to a computer network. The network interface 3300 may connect the consultation information processing unit 3000 to a computer network using a wireless connection or a wired connection method. The consultation information processing unit 3000 may be wirelessly connected or wired to another electronic device or system through the network interface 3300.

The bus may enable communication and data transmission between components of the consultation information processing unit 3000. The bus may be configured using one or more of a high-speed serial bus, a parallel bus, a storage area network (SAN), and other communication methods.

The image processing unit 3110 receives a counselor image from the image capturing unit 2000. The counselor image may be photographed and generated to include the counselor's face. The image processing unit 3110 may extract coordinates of a 3D facial image of the counselor from the counselor image. The coordinates of the 3D facial image of the counselor may be received from the image photographing unit 2000 capable of measuring a distance between the photographing device and each part of a subject photographed in the image. The image processing unit 3110 makes it possible to provide a mapping image in real time by extracting feature points on the face of the counselor in the counselor image in order to easily map the covering image to the counselor image.

The image processing unit 3110 receives the covering image from the counselee terminal 4000. The covering image includes an image of a cover person to be mapped to a counselor in the counselor image. In the covering image, a feature part of the cover person corresponding to all or part of the coordinates of the 3D facial image extracted from the counselor may appear in order to map the counselor image to the counselor. The cover person of the covering image may be a real person who wants to indirectly face the counselee within the environment of the virtual mapping image. The counselee visits the counselor with a covering video in which a person in reality that is difficult to face in person appears, and while watching a mapping image that maps the covering image to the counselor image, indirectly encounters a person who is difficult to face in reality and recognizes the current psychological state from it. , Counseling effects such as recognizing changes in psychological state and improving psychological state can be achieved.

The image processing unit 3110 generates a mapping image by mapping the covering image to the counselor image. The mapping image may be generated by synthesizing a part or all of the image of the counselor into an image included in the covering image based on the part or all of the image of the counselor. The mapping image may be generated by replacing or synthesizing a part or all images of a counselor appearing in the counselor image with a part or all images of a cover person appearing in the covering image. In this way, by mapping the covering image to the counselor image through the image processing unit 3110, when the counselor views the mapping image, it is possible to feel as if he is consulting with the cover person of the covering image provided by the counselor.

The image processing unit 3110 may extract 3D facial coordinate information of the cover person appearing in the covering image. The image information on the covering image includes 3D facial coordinate information for the cover person, which is a person to be mapped to the counselor image among the persons appearing. The image processing unit 3110 may prepare information for mapping the counselor image to the counselor by extracting 3D facial coordinate information of the cover person from the covering image.

The 3D facial coordinate information of the cover person may be extracted from the covering image by the image processing unit 3110 with a predetermined image processing algorithm previously stored in the memory 3200. The 3D facial coordinate information may be extracted through a specific shape, pattern, color, or combination thereof included in the face of the cover person. Image processing algorithms for extracting facial coordinate information include HOG (Histogram of Oriented Gradient), Haar feature, SIFT (Scale Invariant Feature Transform), LBP (Local Binary Pattern), and various other known image processing algorithms. Can be. Based on such an image processing algorithm, coordinate information for one or more of the portions of the face including the eye, nose, mouth, chin, and forehead may be extracted from the face of the cover person. The 3D facial coordinate information of the cover person may include a change value in each frame based on the coordinate information for the reference frame. In addition, the 3D facial coordinate information of the cover person may be obtained by calculating one reference coordinate based on the facial coordinate information secured in each frame.

The image processing unit 3110 may map the cover person to the counselor appearing in the counselor image based on 3D facial coordinate information of the cover person appearing in the covering image. The image processing unit 3110 may replace or synthesize a part or all of the image of the counselor in the image of the counselor with a part or all of the image of the cover person in the covering image based on the 3D facial coordinate information of the cover person and the counselor. The image processing unit 3110 matches each characteristic part based on the 3D facial coordinate information of the cover person and the 3D facial coordinate information of the consultant, and the cover person corresponding to the image according to the facial angle of the consultant in each frame You can create an image of A well-known image synthesis algorithm may be freely employed to synthesize the image of the cover person into the counselor image.

The image processing unit 3110 transmits the generated mapping image to the counselee terminal 4000. The mapping image is transmitted from the consultation information processing unit 3000 to the counselee terminal 4000 after the mapping of the entire image is completed, or the viewable part is sequentially selected so that the counselee can directly view the mapping image while the mapping image is being transmitted. 4000) can also be transmitted in a streaming method.

The reaction information collection unit 3130 receives reaction information of the counselee from the reaction information measurement unit 1000. The response information collection unit 3130 receives pulse information measured by the pulse measurement module 1100, receives brain wave information measured by the brain wave measurement module 1200, and is measured by the motion detection module 1300. Can receive motion information.

The response information collection unit 3130 receives the response information measured by the response information measurement unit 1000 while the mapping image is being played back in the target terminal 4000. The response information includes biometric information such as brain waves, pulse, body temperature, blood sugar, pupil changes, blood pressure, and the amount of dissolved oxygen in the blood that can be measured from the subject's body, and motion information capable of quantifying the detectable movement of the subject. The response information may include one or more of pulse information, brain wave information, and motion information of the subject. The pulse information may include one or more of the heart rate, heart rate variability index, sympathetic activity index, parasympathetic activity index, autonomic activity index, and autonomic balance index. The EEG information may include information on one or more frequency domains of delta, theta, alpha, beta, and gamma measured from the left and right brains. The motion information includes input motions such as selecting some of two or more options or executing a function according to the playback of an image, an intentional expression motion required to proceed with a consultation through an image, the client's concentration, tension, and It may include information on characteristic behaviors of the subject that can be quantified, such as psychological movements to grasp psychological changes such as embarrassment.

The response information analysis unit 3140 generates counseling result information based on the response information. The counseling result information may include analysis information that may be derived based on one or more of the response information, the mapping image, the response information, and the image information of the counselee. Counseling result information may be provided to the counselor, and may include information for confirming what kind of reaction the counselee showed through viewing of the mapping image, and confirming the psychological state of the counselor's response and the contents of the psychological change.

The reaction information analysis unit 3140 may generate analysis information based on the received reaction information. The analysis information may include abnormal pulse information and abnormal EEG information measured as a value outside a predetermined range, and motion information during a time when the abnormal pulse information or the abnormal EEG information is generated. The abnormal pulse information and the abnormal EEG information may be pulse information and brain wave information measured during different sections, or pulse information and EEG information having a difference of more than a preset value compared to the average pulse information and EEG information. The analysis information is information on the psychological state of the subject such as tension, embarrassment, concentration, anxiety, stress, excitement, and tranquility through one or more of pulse information including abnormal pulse information, brain wave information including abnormal brain wave information, and motion information. It may include.

The response information analysis unit 3140 includes a gamma wave indicating an anxiety or excitement state of the subject through brain wave information, a beta wave indicating an activity, stress, or immersion state, an alpha wave indicating a stable state such as meditation or relaxation, and a shallow sleep or sleepiness. Through the theta wave representing the state and the delta wave representing the deep sleep state, the psychological state and its level of the counselee can be derived. The response information analysis unit 3140 can directly or indirectly derive the psychological state and level of the counselee by using the tendency to change irregularly by physical factors such as psychological factors such as anxiety and stress and physical factors such as movement of the counselee through pulse information. have. The type of reaction information required for the determination of each psychological state, the calculation process for determining the level of each psychological state according to the corresponding reaction information, and the numerical range for each level are adopted from known research results, but continuous consultation result information is accumulated. It can be reset through confirmation and feedback of the counselor and condition correction.

The counseling information providing unit 3150 may generate new counseling result information by matching image information on the mapping image with analysis information. The counseling result information may include progress information and results to be provided to the counselee who has undergone psychological counseling, such as response information, video information, analysis information, and information matching such information according to various conditions. The counselee can recognize the psychological state, the factors on the mapping image that have influenced the change in the psychological state, and grasp the change in the psychological state by grasping the change compared to the previous psychological counseling.

The counseling information provider 3150 may add new counseling result information to the existing counseling result information. In the case of a counselor with an existing psychological counseling history, the existing counseling result information exists in the counselor information 3230 stored in the memory 3200 of the counseling information processing unit 3000, so when providing psychological result information, it is provided together with the existing counseling result information. Also, the counselor may add new counseling result information to the existing counseling result information of the counselee information 3230 to continue management.

The counseling information provider 3150 transmits the counseling result information to the counselee terminal 4000. The counseling information providing unit 3150 may transmit counseling result information in which the existing counseling result information and the new counseling result information are combined to the counselee terminal 4000. The counseling information providing unit 3150 may also transmit counseling result information to the counselor terminal 5000 so that the counselor may also check the counseling result information. The counseling result information transmitted to the counselee terminal 4000 may be provided by visualization so that the counselee can easily grasp response information, motion information, analysis information, etc. in addition to counseling result information for management.

4, a psychological counseling method capable of personalized video counseling through real-time image mapping according to an embodiment of the present invention using the aforementioned psychological counseling system includes a counselor image receiving step (S100) and a covering image receiving step ( S200), mapping image generation step (S300), mapping image transmission step (S400), response information receiving step (S500), counseling result information generating step (S600), and counseling result information transmitting step (S700). Include.

First, a counselor image, which is the basis of the mapping image, is received from the image photographing unit 2000 (S100). The counselor's image may include the counselor's face. The counselor image may include information on coordinates of the counselor's 3D facial image. In order to easily map the covering image to the counselor image, feature points on the face of the counselor in the counselor image are extracted, thereby making it possible to provide the mapping image in real time.

Then, the covering image is received from the target terminal (S200). The covering image includes an image of a cover person to be mapped to the counselor of the counselor image. In the covering image, a feature part of the cover person corresponding to all or part of the coordinates of the 3D facial image extracted from the counselor may appear in order to map the counselor image to the counselor. The cover person of the covering image may be a real person who wants to indirectly face the counselee within the environment of the virtual mapping image. The counselee visits the counselor with a covering video of a person who is difficult to face in person, and while watching the mapping image that maps the covering image to the counselor image where the counselor appears, indirectly feels a situation that is difficult to face in reality and from there Counseling effects such as recognizing the psychological state of the patient, recognizing the change in the mental state, and improving the mental state can be achieved.

Next, a mapping image is generated by mapping the covering image to the counselor image (S300). The mapping image may be generated by synthesizing a part or all of the image of the counselor into an image included in the covering image based on the part or all of the image of the counselor. The mapping image may be generated by replacing or synthesizing some or all images of the counselor with some or all images of the cover person appearing in the covering image. In this way, by mapping the covering image to the counselor image, when the counselor views the mapping image, it is possible to feel as if they are consulting with the cover person of the covering image provided by the counselor. The mapping image generation step S300 may include the following specific steps.

First, it is possible to extract 3D facial coordinate information of the cover person appearing in the covering image. The image information on the covering image includes 3D facial coordinate information for the cover person, which is a person to be mapped to the counselor image among the persons appearing. Information for mapping the counselor image to the counselor may be prepared by extracting 3D facial coordinate information of the cover person from the covering image secured in the receiving step S200.

The 3D facial coordinate information of the cover person may be extracted from the covering image through the image processing algorithm described above. The 3D facial coordinate information may be extracted through a specific shape, pattern, color, or combination thereof included in the face of the cover person. The 3D facial coordinate information of the cover person may be extracted from the covering image based on an angle obtained by obtaining the counselor's facial coordinates in the reference frame of the counselor image. In addition, the 3D facial coordinate information of the cover person may be obtained by calculating one reference coordinate based on the facial coordinate information secured in each frame.

Next, the cover person may be mapped to the counselor appearing in the counselor image based on 3D facial coordinate information of the cover person appearing in the covering image. On the basis of the cover person and the 3D facial coordinate information of the counselor, a part or the entire image of the counselor in the counselor image may be replaced or synthesized with a part or all image of the cover person in the covering image. Based on the 3D facial coordinate information of the cover person and the 3D facial coordinate information of the consultant, each characteristic part is matched, and an image of the cover person corresponding to the image according to the facial angle of the consultant can be generated in each frame. have. A well-known image synthesis algorithm may be freely employed to synthesize the image of the cover person into the counselor image.

Then, the generated mapping image is transmitted to the target terminal (S400). The mapping image may be transmitted to the client terminal after the mapping of the entire image is completed, or may be transmitted in a streaming manner so that the viewable portion is sequentially transmitted to the client terminal so that the client can immediately view the mapping image while the mapping image is being transmitted.

Then, the measured response information is received while the mapping image is being reproduced in the subject terminal (S500). The response information includes biometric information such as brain waves, pulse, body temperature, blood sugar, pupil changes, blood pressure, and the amount of dissolved oxygen in the blood that can be measured from the subject's body, and motion information capable of quantifying the detectable movement of the subject. The response information may include one or more of pulse information, brain wave information, and motion information of the subject. The pulse information may include one or more of the heart rate, heart rate variability index, sympathetic activity index, parasympathetic activity index, autonomic activity index, and autonomic balance index. The EEG information may include information on one or more frequency domains of delta, theta, alpha, beta, and gamma measured from the left and right brains. The motion information includes input motions such as selecting some of two or more options or executing a function according to the playback of an image, an intentional expression motion required to proceed with a consultation through an image, the client's concentration, tension, and It may include information on characteristic behaviors of the subject that can be quantified, such as psychological movements to grasp psychological changes such as embarrassment.

Next, the consultation result information is generated based on the reaction information (S600). The counseling result information may include analysis information that may be derived based on one or more of response information of the counselee, detailed image information about the counselor image used in the mapping image and the mapping image, response information, and image information. Counseling result information may be provided to the counselor, and may include information for confirming what kind of reaction the counselee showed through viewing of the mapping image, and confirming the psychological state of the counselor's response and the contents of the psychological change. The step of generating counseling result information (S600) may include the following specific steps.

Referring to FIG. 5, first, analysis information may be generated based on the received reaction information (S610). The analysis information may include abnormal pulse information and abnormal EEG information measured as a value outside a predetermined range, and motion information during a time when the abnormal pulse information or the abnormal EEG information is generated. The abnormal pulse information and the abnormal EEG information may be pulse information and brain wave information measured during different sections, or pulse information and EEG information having a difference of more than a preset value compared to the average pulse information and EEG information. The analysis information is information on the psychological state of the subject such as tension, embarrassment, concentration, anxiety, stress, excitement, and tranquility through one or more of pulse information including abnormal pulse information, brain wave information including abnormal brain wave information, and motion information. It may include.

According to the EEG information, gamma waves indicating anxiety or excitement of the subject, beta waves indicating activity, stress, or immersion, alpha waves indicating stable states such as meditation or relaxation, theta waves indicating shallow sleep or sleepiness, and deep sleep. Through the delta wave representing the state, the psychological state of the counselee and its level can be derived. In addition, according to the pulse rate information, it is possible to directly or indirectly derive the psychological state and level of the counselee by using the tendency to change irregularly by physical factors such as psychological factors such as anxiety and stress, and physical factors such as movement of the counselee. The type of reaction information required for the determination of each psychological state, the calculation process for determining the level of each psychological state according to the corresponding reaction information, and the numerical range for each level are adopted from known research results, but continuous consultation result information is accumulated. It can be reset through confirmation and feedback of the counselor and condition correction.

The analysis information may include analysis information on a relationship between the response information and the image information according to time while the counselee views the mapping image. The analysis information may include information on a psychological state determined according to the type and level of abnormal pulse information, abnormal brain wave information, or other reaction information and the level of change thereof. By synchronizing the playback time of the mapping image and the reception time of the response information, the state and change of the response information according to the image information by time can be checked through the analysis information. Through this, the counselee can recognize the change in the psychological state and the psychological state, and can grasp the change in the psychological state by grasping the change between each time point during the corresponding psychological counseling and the change compared to the previous psychological counseling.

Next, new consultation result information may be generated by matching the image information on the mapping image with the analysis information (S620). The counseling result information may include progress information and results to be provided to the counselee who has undergone psychological counseling, such as response information, video information, analysis information, and information matching such information according to various conditions. The consultation result information may include a content that determines whether a meaningful relationship exists between the analysis information generated in the analysis information generation step S620 and the features that have influenced the corresponding information. The counselee can recognize the psychological state and the factors that have affected the change in the psychological state, and can grasp the change in the psychological state by grasping the change between each time point during the corresponding psychological counseling and the change compared to the previous psychological counseling.

Next, the new consultation result information may be added to the existing consultation result information (S630). In the case of a counselor with an existing psychological counseling history, the existing counseling result information exists. Therefore, when providing the psychological result information, new counseling result information is added to the existing counseling result information in order to provide the information along with the existing counseling result information and continue to manage the counselor. can do.

Then, the counseling result information is transmitted to the terminal of the counselee (S700). In the step of adding counseling result information (S630), the counseling result information obtained by integrating the existing counseling result information and the new counseling result information may be transmitted to the client terminal. Counseling result information transmitted to the counselee terminal may be provided by visualization so that the counselee can easily grasp reaction information, analysis information, etc. along with counseling result information for management.

According to the psychological counseling method according to the above-described embodiment, by providing personalized counseling through an image in which the counselor's desired covering image is mapped in real time to the counselor image, the counselor can participate in the counseling with a more comfortable mind, and at the same time, brain waves and pulse rate. By collecting and analyzing the reaction information of the counselee, such as, motion, etc., it is possible to grasp the factors and patterns of change in the mental state. In particular, the counselee can simultaneously check his/her body changes while experiencing the situation of indirectly contacting the cover person in the covering image that he wants to face through the mapping image. It will be possible to recognize change patterns, internal and external factors, etc., and will be able to identify psychological changes in repeated psychological counseling and seek directions for improving mental state.

The psychological counseling method implemented by the psychological counseling system capable of personalized video counseling through real-time image mapping according to the above embodiment may be implemented by the computing device 9000 illustrated in FIG. 6. The consultation information processing unit 3000 of FIG. 6 may include components of the computing device 9000 illustrated in FIG. 7.

The computing device 9000 may include one or more processors 9100, a memory 9200, a peripheral interface 9300, an input/output subsystem 9400, a power circuit 9500, and a communication circuit 9600. . The computing device 9000 includes part or all of the response information measurement unit 1000, the image capture unit 2000, the consultation information processing unit 3000, the counselee terminal 4000, and the counselor terminal 5000, or input/output The subsystem 9400 may be connected to a part or all of the response information measuring unit 1000, the image capturing unit 2000, the counseling information processing unit 3000, the consultant terminal 4000, and the consultant terminal 5000.

The memory 9200 may include, for example, high-speed random access memory, magnetic disk, SRAM, DRAM, ROM, flash memory, or nonvolatile memory. The memory 9200 may include software modules, instruction sets, or other data necessary for the operation of the computing device 9000. Access to the memory 9200 from a component such as the processor 9100 or the peripheral interface 9300 may be controlled by the processor 9100.

The peripheral device interface 9300 may connect one or more of an input peripheral device and an output peripheral device of the computing device 9000 to the processor 9100 and the memory 9200. The processor 9100 may perform various functions for the computing device 9000 and process data by executing a software module or instruction set stored in the memory 9200.

The input/output subsystem 9400 may couple various input/output peripherals to the peripherals interface 9300. For example, the input/output subsystem 9400 may include a controller for coupling peripheral devices such as a monitor, keyboard, mouse, printer, touch screen, sensor, and the like to the peripheral device interface 9300. The input/output peripherals may be coupled to the peripherals interface 9300 without going through the input/output subsystem 9400.

The power circuit 9500 may supply power to all or part of the components of the terminal. For example, the power circuit 9500 may be a power management system, one or more power sources such as batteries or alternating current (AC), a charging system, a power failure detection circuit, a power converter or inverter, a power status indicator or power. It may contain any other components for creation, management, and distribution.

The communication circuit 9600 may enable communication with other computing devices using one or more external ports. The communication circuit 9600 may include an RF circuit to transmit and receive RF signals to enable communication with other computing devices.

As an example, the computing device 9000 may have a configuration or arrangement in which some illustrated components are omitted, an additional component not illustrated is further provided, or two or more components are combined. For example, a computing device for a communication terminal in a mobile environment may further include a touch screen or a sensor, and various communication methods (WiFi, 3G, LTE, Bluetooth, NFC, Zigbee, etc.) are used in the communication circuit 9600. Circuitry for RF communication may also be included. Components that may be included in the computing device 9000 may be implemented in hardware, software, or a combination of both hardware and software including one or more signal processing or application-specific integrated circuits.

Methods according to an embodiment of the present invention may be implemented in the form of program instructions that can be executed through various computing devices and recorded in a computer-readable medium. In particular, the program according to the present embodiment may be configured as a PC-based program or an application dedicated to a mobile terminal. An application to which the present invention is applied may be installed on a user terminal through a file provided by the file distribution system. For example, the file distribution system may include a file transmission unit (not shown) that transmits the file according to the request of the user terminal.

The apparatus described above may be implemented as a hardware component, a software component, and/or a combination of a hardware component and a software component. For example, the devices and components described in the embodiments are, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA). , A programmable logic unit (PLU), a microprocessor, or any other device capable of executing and responding to instructions, such as one or more general purpose computers or special purpose computers. The processing device may execute an operating system (OS) and one or more software applications executed on the operating system. In addition, the processing device may access, store, manipulate, process, and generate data in response to the execution of software. For the convenience of understanding, although it is sometimes described that one processing device is used, one of ordinary skill in the art, the processing device is a plurality of processing elements and/or a plurality of types of processing elements. It can be seen that it may include. For example, the processing device may include a plurality of processors or one processor and one controller. In addition, other processing configurations are possible, such as a parallel processor.

The software may include a computer program, code, instructions, or a combination of one or more of these, configuring the processing unit to behave as desired or processed independently or collectively. You can command the device. Software and/or data may be interpreted by a processing device or to provide instructions or data to a processing device, of any type of machine, component, physical device, virtual equipment, computer storage medium or device. , Or may be permanently or temporarily embodyed in a transmitted signal wave. The software may be distributed over networked computing devices and stored or executed in a distributed manner. Software and data may be stored on one or more computer-readable recording media.

The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded in a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, etc. alone or in combination. The program instructions recorded on the medium may be specially designed and configured for the embodiment, or may be known and usable to those skilled in computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CDROMs and DVDs, and magnetic-optical media such as floptical disks. And hardware devices specially configured to store and execute program instructions such as magneto-optical media, and ROM, RAM, flash memory, and the like. Examples of the program instructions include not only machine language codes such as those produced by a compiler, but also high-level language codes that can be executed by a computer using an interpreter or the like. The hardware device described above may be configured to operate as one or more software modules to perform the operation of the embodiment, and vice versa.

The communication connection in the system and method according to the embodiments of the present invention may be configured regardless of a communication mode such as wired communication or wireless communication, and a local area network (LAN) or a metropolitan area communication network (MAN; Metropolitan Area) Network), a wide area network (WAN), and the like. Preferably, the communication connection in this specification may be a publicly known Internet or World Wide Web (WWW). However, the communication connection is not necessarily limited thereto, and may include a known wired/wireless data communication network, a known telephone network, or a known wired/wireless television communication network in at least part thereof. For example, the communication connection is a wireless data communication network, such as WiFi communication, WiFi-Direct communication, Long Term Evolution (LTE) communication, Bluetooth communication (more specifically, low-power Bluetooth (BLE) communication). ; Bluetooth Low Energy)), infrared communication, ultrasonic communication, and the like may be implemented in at least part of a conventional communication method. For another example, the communication connection is an optical communication network, and a conventional communication method such as LiFi (Light Fidelity) may be implemented in at least a portion thereof.

Although the embodiments of the present invention have been described in more detail with reference to the accompanying drawings, the present invention is not necessarily limited to these embodiments, and various modifications can be made without departing from the spirit of the present invention. . Accordingly, the embodiments disclosed in the present invention are not intended to limit the technical idea of the present invention, but to explain the technical idea, and the scope of the technical idea of the present invention is not limited by these embodiments. Therefore, it should be understood that the embodiments described above are illustrative in all respects and not limiting. The scope of protection of the present invention should be interpreted by the following claims, and all technical ideas within the scope equivalent thereto should be interpreted as being included in the scope of the present invention.

1000: reaction information measuring unit
1100: pulse measurement module
1200: EEG measurement module
1300: motion detection module
2000: video recording department
3000: consultation information processing unit
4000: counselee terminal
5000: consultant terminal

Claims (3)

As a psychological counseling method that enables personalized video counseling through real-time image mapping,
Receiving an image of a counselor from an image photographing unit;
Receiving a covering image from a target terminal;
Generating a mapping image by mapping the covering image to the counselor image;
Transmitting the mapping image to the counselee terminal;
Receiving reaction information of a subject from a response information measuring unit while the mapping image is being reproduced in the subject terminal;
Generating counseling result information based on the response information; And
Transmitting the counseling result information to the counselee terminal;
Including,
The step of generating the mapping image,
Extracting 3D facial coordinate information of a cover person appearing in the covering image; And
Mapping the cover person to the counselor appearing in the counselor image based on the 3D facial coordinate information of the cover person appearing in the covering image;
Including,
Generating the consultation result information,
Generating analysis information based on the reaction information;
Generating new consultation result information by matching image information on the mapping image with the analysis information;
Generating the counseling result information including existing counseling result information and the new counseling result information; And
Adding the new consultation result information to the existing consultation result information;
Including,
The reaction information includes at least one of pulse information, brain wave information, and motion information of the subject,
The analysis information includes abnormal pulse information and abnormal EEG information, the abnormal pulse information, or motion information during a time when the abnormal EEG information is generated, measured as a value outside a predetermined range,
The analysis information is information on the relationship between the response information and the image information according to time while the counselee views the mapping image through synchronization of the playback time of the mapping image and the reception time of the response information A psychological counseling method that includes information on the factors on the mapping image that have influenced the change of the psychological state and the change in the psychological state of the counselee between points during psychological counseling.

delete delete

Images