WO2023018325A1 - Systèmes et procédés pour conduire et évaluer des sessions de psychothérapie à distance - Google Patents

Systèmes et procédés pour conduire et évaluer des sessions de psychothérapie à distance Download PDF

Info

Publication number
WO2023018325A1
WO2023018325A1 PCT/MY2022/050047 MY2022050047W WO2023018325A1 WO 2023018325 A1 WO2023018325 A1 WO 2023018325A1 MY 2022050047 W MY2022050047 W MY 2022050047W WO 2023018325 A1 WO2023018325 A1 WO 2023018325A1
Authority
WO
WIPO (PCT)
Prior art keywords
client
module
indication
therapist
paralinguistic
Prior art date
Application number
PCT/MY2022/050047
Other languages
English (en)
Inventor
Tiffanie Ru Yi ONG
Jehanita JESUTHASAN
Original Assignee
Naluri Hidup Sdn Bhd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Naluri Hidup Sdn Bhd filed Critical Naluri Hidup Sdn Bhd
Publication of WO2023018325A1 publication Critical patent/WO2023018325A1/fr

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00—Handling natural language data
    • G06F40/30—Semantic analysis
    • G06F40/35—Discourse or dialogue representation
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/0002—Remote monitoring of patients using telemetry, e.g. transmission of vital signals via a communication network
    • A61B5/0015—Remote monitoring of patients using telemetry, e.g. transmission of vital signals via a communication network characterised by features of the telemetry system
    • A61B5/0022—Monitoring a patient using a global network, e.g. telephone networks, internet
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/16—Devices for psychotechnics; Testing reaction times ; Devices for evaluating the psychological state
    • A61B5/165—Evaluating the state of mind, e.g. depression, anxiety
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/16—Devices for psychotechnics; Testing reaction times ; Devices for evaluating the psychological state
    • A61B5/167—Personality evaluation
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
    • A61B5/7235—Details of waveform analysis
    • A61B5/7264—Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/174—Facial expression recognition
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/20—Movements or behaviour, e.g. gesture recognition
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L25/00—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
    • G10L25/48—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use
    • G10L25/51—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use for comparison or discrimination
    • G10L25/63—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use for comparison or discrimination for estimating an emotional state
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L25/00—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
    • G10L25/48—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use
    • G10L25/51—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use for comparison or discrimination
    • G10L25/66—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use for comparison or discrimination for extracting parameters related to health condition
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H20/00—ICT specially adapted for therapies or health-improving plans, e.g. for handling prescriptions, for steering therapy or for monitoring patient compliance
    • G16H20/70—ICT specially adapted for therapies or health-improving plans, e.g. for handling prescriptions, for steering therapy or for monitoring patient compliance relating to mental therapies, e.g. psychological therapy or autogenous training
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H50/00—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics
    • G16H50/30—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics for calculating health indices; for individual health risk assessment
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H80/00—ICT specially adapted for facilitating communication between medical practitioners or patients, e.g. for collaborative diagnosis, therapy or health monitoring
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00—Handling natural language data
    • G06F40/20—Natural language analysis
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00—Handling natural language data
    • G06F40/20—Natural language analysis
    • G06F40/253—Grammatical analysis; Style critique
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00—Handling natural language data
    • G06F40/30—Semantic analysis
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00—Speech recognition
    • G10L15/26—Speech to text systems
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H40/00—ICT specially adapted for the management or administration of healthcare resources or facilities; ICT specially adapted for the management or operation of medical equipment or devices
    • G16H40/60—ICT specially adapted for the management or administration of healthcare resources or facilities; ICT specially adapted for the management or operation of medical equipment or devices for the operation of medical equipment or devices
    • G16H40/67—ICT specially adapted for the management or administration of healthcare resources or facilities; ICT specially adapted for the management or operation of medical equipment or devices for the operation of medical equipment or devices for remote operation

Definitions

  • the invention relates to psychotherapy.
  • the invention pertains to systems and methods for conducting and assessing remote psychotherapy sessions.
  • Psychotherapy and counseling sessions have traditionally been conducted in a therapist’s office, where a patient may communicate and convey several information including the patient’s experiences, feelings, history and other personal information to a therapist. Based on the information revealed by the patient, the therapist may provide appropriate therapy or counseling to the patient.
  • the internet technology has emerged as a global means of communication in the modem lifestyles, it has also been used as one of the modes for delivering the psychotherapy and counseling services to one or more patients.
  • PCT Publication No. WO 2016/071660 discloses a computer-based system for providing psychological therapy.
  • the system includes a number of remote computing devices connected to a server via one or more network systems, where the server includes several system modules, preferably including an access system for controlling access to features and data by users of the remote devices (such as patients, therapists and supervisors) and a therapy system for enabling text-based instant messages to be sent between the patients and therapists.
  • the access system is also configured to allow the patients to retrieve messages sent and received by the patient, the therapists to retrieve messages sent and received by the therapist, and the supervisors to retrieve messages sent and received by particular patients, therapists or both.
  • U.S. Patent Publication No. US 2012/0320145 also discloses methods and systems for online counseling sessions conducted over the internet.
  • a method for remotely conducting counseling sessions between a client and an expert using an IP -based network includes providing a website accessible to the network wherein the expert and a client both have access to the website.
  • the client upon accessing and logging into the website, the client is permitted to search a database for online experts and select an appropriate expert for a counseling session.
  • the client initially completes various intake and/or registration forms in a virtual waiting room, wherein such forms are customized as per the relevant expert.
  • the method continues as an expert remotely conducts a counseling session with the client via remote means, including video conferencing. Following the counseling session, the client is automatically returned to the virtual waiting room and provided with subsequent counseling-based options.
  • the internet enables the patients or general public to gain access to the psychotherapy and counseling services from home within minutes and cost effectively, there are inherent limitations in delivering these services via the internet.
  • the internet psychotherapy and counseling sessions are more prone to miscommunications, as certain information including actual thoughts and feelings of the patients may be lost or inadequately conveyed during the online communications between the patients and the therapists.
  • text-based communications such as text messages and emails often lack the non-verbal cues associated with emotions.
  • the patients or therapists may also lack the writing skills to adequately express themselves. Consequently, the messages delivered may be incorrectly interpreted by the therapists or patients, thereby affecting the effectiveness of the therapy or counseling sessions and possibly jeopardizing the relationship between the patients and the therapists.
  • One of the objects of the invention is to provide improved systems and methods for remotely conducting psychotherapy between a therapist and one or more clients.
  • Another object of the invention is to provide systems and methods enabling a plurality of data to be extracted and analysed in real-time or near real-time during each remote psychotherapy session, in order to generate indications which are insightful and helpful to the therapist in assessing the psychotherapy.
  • Still another object of the invention is to introduce systems and methods enabling the therapist or therapists to review results generated after each psychotherapy session.
  • one of the embodiments of the invention describes a system for conducting a remote psychotherapy between a therapist provided with a therapist computing device and at least a client provided with a client computing device, the system comprising a Natural Language Processing (NLP) module for converting the therapist’s speeches and client’s speeches captured and recorded during a two-way audio-video communication session established between the computing devices to text-based speech data, and then analyse the text-based speech data, to generate at least a first indication associated with a degree of verbal concordance between the therapist and client, and a second indication associated with an insight data in response to a therapist-client scenario; a paralinguistic feature analysing module for converting the therapist’s paralinguistic features and the client’s paralinguistic features captured and recorded during the communication session to paralinguistic data, and then analyse the paralinguistic data, to generate at least a third indication associated with the client’s psychological state, and a fourth indication associated with the a degree
  • NLP Natural Language Processing
  • the paralinguistic features captured and recorded may comprise body paralinguistic features and vocal paralinguistic features.
  • the therapy assessment module may be configured to receive the first indication, second indication, third indication and fourth indication from the NLP module and paralinguistic feature analysing module after the psychotherapy has ended, and then assess the psychotherapy based on the indications by determining at least a first result in relation to therapeutic relationship between the therapist and the client, based on the first indication and fourth indication; a third result related to whether the client is honest or dishonest, based on the second indication and third indication; and a fourth result related to whether a therapeutic approach is effective, based on the second indication.
  • the system may also comprise a baseline state analysing module configured to request for the client’s reactions to a list of questions at the beginning of the psychotherapy and then determine a baseline indication associated with the client’s baseline state based on the client’s reactions.
  • the client’s reactions herein may include speeches, body paralinguistic features and vocal paralinguistic features, which may be analysed by the NLP module and paralinguistic feature analysing module.
  • the NLP module may be further configured to generate a fifth indication associated with a degree of verbal concordance between any two clients, and the paralinguistic feature analysing module may also be further configured to generate a sixth indication associated with a degree of non-verbal concordance between any two clients.
  • the therapy assessment module may further be configured to determine a second result in relation to the client’s emotional valence score, based on the second indication, third indication and baseline indication; and determine a fifth result in relation to the therapeutic relationship between the two clients, based on the fifth indication and sixth indication.
  • the system may further comprise a feedback module in data communication with the NLP module and the therapy assessment module.
  • the feedback module may also be configured to allow the therapist to review and provide feedbacks in relation to the textbased speech data, indications and results received from the NLP module and therapy assessment module respectively.
  • system may further comprise one or more databases to store data received from the NLP module, paralinguistic feature analysing module, baseline state analysing module and therapy assessment module.
  • a further embodiment of the invention is a method to conduct a remote psychotherapy between a therapist provided with a therapist computing device and at least a client provided with a client computing device, comprising the steps of establishing a two- way audio-video communication session between the therapist computing device and client computing device, wherein the computing devices are each operatively coupled to the internet; capturing the therapist’s reactions and the client’s reactions during the communication session by using the computing devices, wherein the reactions to be captured may be speeches, body paralinguistic features, vocal paralinguistic features or a combination thereof; converting the captured speeches to text-based speech data using a NLP module, if the therapist’s and the client’s speeches are captured; analysing the text-based speech data by using the NLP module, to generate at least a first indication associated with a degree of verbal concordance between the therapist and the client, and a second indication associated with an insight data in response to a therapist-client scenario; converting the captured paralinguistic features to a plurality of paralinguistic data by using a
  • the method may further comprise the steps of receiving the client’s reactions to a list of questions by a baseline state analysing module at the beginning of the psychotherapy, and determining a baseline indication associated with the client’s baseline state based on the client’s reactions, by using the baseline state analysing module.
  • the method may further comprise the step of determining a second result in relation to the client’s emotional valence score, based on the second indication, third indication and baseline indication. If more than one client is engaged in the psychotherapy, the method may further comprise the steps of determining a fifth indication associated with a degree of verbal concordance between a pair of selected clients, by using the NLP module; and determining a sixth indication associated with a degree of non-verbal concordance between the selected clients, using the paralinguistic feature analysing module.
  • the method may further comprise the steps of receiving different types of inputs through a user interface of the therapist computing device, after reviewing the text-based speech data, indications and results.
  • the method may further comprise the steps of storing a plurality of data received from the NLP module (104), paralinguistic feature analysing module (106), baseline state analysing module (108) and therapy assessment module (110) respectively.
  • the method depicted in the foregoing may alternatively be stored and implemented as a series of instructions executable by either a computing device or a processing module of the computing device.
  • Figure 1 shows a system for conducting and assessing a remote psychotherapy session between a therapist and a client, according to one embodiment.
  • Figure 2 shows a system for conducting and assessing a remote psychotherapy session between a therapist and more than a client, according to another embodiment.
  • Figure 3 shows a general flow chart of a method for conducting and assessing a remote psychotherapy session between a therapist and a client, according to one embodiment.
  • Figure 4 shows a general flow chart detailing steps to be performed for reviewing data generated in relation to a remote psychotherapy session between a therapist and a client, according to one embodiment.
  • Figure 5 shows a general flow chart of a method for conducting and assessing a remote psychotherapy session between a therapist and more than a client, according to another embodiment.
  • Figure 6 shows a general flow chart detailing steps to be performed for reviewing data generated in relation to a remote psychotherapy session between a therapist and more than a client, according to another embodiment.
  • client and “patient” are used interchangeably throughout the specification to refer to a recipient of a remote psychotherapy session.
  • the invention provides systems and methods for remotely conducting psychotherapy between a therapist and one or more clients.
  • the systems and methods may also require that a plurality of data be extracted and analysed in real-time or near real-time during each remote psychotherapy session, in order to generate indications which are insightful and helpful to the therapist in assessing the psychotherapy session.
  • Figure 1 shows a system for conducting a psychotherapy session remotely and over the internet between a therapist and a client, according to one embodiment of the invention.
  • the therapist is provided with a therapist computing device (102a) and the client is provided with a client computing device (102b).
  • the computing devices (102a, 102b) may each be a computing device operable to establish an internet connection and access a web application or web browsing application, such as a desktop computer, a laptop computer, a tablet computer or a mobile smart phone, but each computing device (102a, 102b) may preferably be provided with at least a video camera to record an image of a person, a microphone to record sound of a person and a speaker to play audio to a person.
  • Each computing device (102a, 102b) may further be configured to capture and record the therapist’s reactions and the client’s reactions during the psychotherapy session by using the video camera and microphone.
  • the reactions to be captured and recorded may comprise one or more of the following: the therapist’s speeches, the client’s speeches, the therapist’s body paralinguistic features (such as the therapist’s facial expressions, eye movements, gestures and postures), the client’s body paralinguistic features (such as the client’s facial expressions, eye movements, gestures and postures), the therapist’s vocal paralinguistic features (such as the therapist’s pitch, loudness or sound pressure, timbre and tone) and the client’s vocal paralinguistic features (such as the client’s pitch, loudness or sound pressure, timbre and tone).
  • the reactions may be recorded in the form of either an audio file or a video file.
  • the speeches and vocal paralinguistic features of Client Ai may be recorded in the form of an audio file
  • the body paralinguistic features of Client Ai may be recorded in the form of a video file.
  • further processing may be required, which will be described in the following.
  • the audio and video files may be processed to provide the therapist with insightful and helpful indications in assessing the psychotherapy session including the therapeutic relationship or alliance between the therapist and client.
  • the system may be provided with several processing modules, such as a natural language processing (NLP) module (104) and a paralinguistic analysing module (106).
  • NLP natural language processing
  • the system may also be configured to enable data transmission or communication between the computing devices (102a, 102b), NLP module (104) and paralinguistic analysing module (106).
  • the NLP module (104) may be configured to receive the audio files from the therapist computing device (102a) and client computing device (102b) in real-time or near real-time during a psychotherapy session.
  • the NLP module (104) may also be configured to convert the speeches in the audio files to text-based speech data and analyse the speech data to generate at least a first indication and a second indication.
  • the first indication may be associated with a degree of verbal concordance between the therapist and the client, whereas the second indication may be associated with an insight data in response to a particular therapist-client scenario.
  • the paralinguistic feature analysing module (106) may be configured to receive the audio files comprising the vocal paralinguistic features and video files comprising the body paralinguistic features from the therapist computing device (102a) and client computing device (102b) during the psychotherapy session, in real-time or near realtime.
  • the paralinguistic feature analysing module (106) may be configured to convert the paralinguistic features to a plurality of paralinguistic data and subsequently analyse the paralinguistic data, in order to generate at least a third indication associated with a psychological state of the client and a fourth indication associated with a degree of nonverbal concordance between the therapist and the client.
  • the psychological state may be a mental state (such as fatigue, sleepiness, concentration, etc.), emotional state (such as anger, fear, sadness, frustration, etc.) or a combination of both.
  • the psychological state may be the client’s concentration level.
  • the paralinguistic feature analysing module (106) may be made up of a body paralinguistic feature analysing submodule (to analyse only body paralinguistic features) and a vocal paralinguistic feature analysing sub-module (to analyse only vocal paralinguistic features), the sub-modules being capable of establishing a data communication with each other.
  • the system may also comprise a baseline state analysing module (108) configured to be in data communication with the client computing device (102b), NLP module (104) and paralinguistic feature analysing module (106).
  • the baseline state analysing module (108) may be configured to determine a baseline indication associated with the client’s baseline state (or “initial state”) at the beginning of the psychotherapy session. In order to determine the baseline indication, the baseline state analysing module (108) may prompt and invite the client to react to a list of questions displayed to the client through the client computing device (102b) or posed to the client by the therapist at the beginning of the psychotherapy session.
  • the client’s reactions in response to the list of questions may be captured and recorded by the client computing device (102b).
  • the reactions to be captured and recorded may comprise the client’s speeches, the client’s body paralinguistic features (such as the client’s facial expressions, eye movements, gestures and postures) and the client’s vocal paralinguistic features (such as the client’s pitch, loudness or sound pressure, timbre and tone).
  • the client’s reactions may also be analysed using the NLP module (104) and paralinguistic feature analysing module (106). Subsequently, based on the analyses performed by the NLP module (104) and paralinguistic feature analysing module (106), the baseline state analysing module (108) may be configured to determine a baseline indication associated with the client’s baseline state.
  • the baseline indication may further be processed to assess fluctuations in the emotional valence of the client during the psychotherapy session. If more than one psychotherapy session may be conducted, the baseline indication for each of the psychotherapy session may be recorded and accumulated, thus allowing for monitoring of the client’s overall progress and the effectiveness of the therapeutic treatment plan. Accumulation of the baseline indications may also enable changes in the psychological or emotional states of the same client to be tracked in the present psychotherapy session and across multiple psychotherapy sessions.
  • the system may also comprise a therapy assessment module (110) configured to be in data communication with the NLP module (104), the paralinguistic feature analysing module (106) and the baseline state analysing module (108) to receive the first, second, third, fourth and baseline indications.
  • the therapy assessment module may also be configured to assess the psychotherapy session upon processing these indications.
  • the therapy assessment module (110) may be configured to determine one or more of the following: a first result in relation to the therapeutic relationship or alliance between the therapist and the client, based on the first and fourth indications; a second result in relation to the client’s emotional valence score, based on the second indication, third indication and the client’s baseline indication; a third result related to whether the client is honest or dishonest, based on the second indication and third indication; and a fourth result related to whether the therapeutic approach is effective, based on the second indication.
  • the system may also comprise a database (120).
  • the database (120) may be configured to be in data communication with the NLP module (104), the paralinguistic feature analysing module (106), the baseline state analysing module (108) and the therapy assessment module (110) respectively.
  • the database (120) may also be configured to store a plurality of data, for instance but not limited to the audio files, video files, text-based speech data, body paralinguistic data, vocal paralinguistic data, indications and results generated.
  • the data may be sorted and stored in the database (120) according to the name of the client. For instance, all data pertaining to a psychotherapy session between Therapist Ai and Client Ai may be stored under a record belonging to Client Ai.
  • the record of Client Ai may comprise one or more of the following: Therapist Ai’s audio files, Therapist Ai’s video files, Therapist Ai’s speech data, Therapist Ai’s body paralinguistic data, Therapist Ai’s vocal paralinguistic data, Client Ai’s audio files, Client Ai’s video files, Client Ai’s speech data, Client Ai’s body paralinguistic data, Client Ai’s vocal paralinguistic data, indications and results generated.
  • a database 120 may be provided depending on the system configuration.
  • the system may further comprise a feedback module (130) in data communication with the NLP module (104) and therapy assessment module (110). Presence of the feedback module (130) may enable a feedback loop to be established, hence allowing continuous training, retraining or fine-tuning to the steps, operations or algorithms to be executed. Thereby, accuracy of the analyses and results could be improved.
  • the feedback module (130) may be configured to mitigate issues caused by the recorded speeches which are of poor quality or unclear.
  • the NLP module (104) may encounter difficulties in identifying the correct words based on the recorded speeches.
  • the text-based speech data produced by the NLP module (104) may be erroneous and inaccurate, which affects the assessment of the psychotherapy session or in particular the performance of the therapy assessment module (110).
  • the feedback module (130) may be configured to transfer and display the text-based speech data to the therapist for viewing via a user interface on a display device of the therapist computing device (102a) after the psychotherapy session has ended.
  • the user interface may also allow the therapist to provide a confirmation input that the text-based speech data do not require corrections.
  • the feedback module (130) may be configured to transmit a signal to the NLP module (104), notifying that the NLP module (104) may upload the originally generated speech data to the database (120).
  • the feedback module (130) may enable the therapist to provide a correction input through the user interface mentioned in the foregoing, to correct the errors in the speech data. Subsequently, the feedback module (130) may transfer the corrected speech data to the NLP module (104), so that the first and second indications may be updated by the NLP module (104) according to the corrected speech data, before transferring the updated indications to the therapy assessment module (110) to update the results and/or before uploading the latest information (e.g. the corrected speech data, updated indications and updated results) to the database (120).
  • the latest information e.g. the corrected speech data, updated indications and updated results
  • the feedback module (130) may be further configured to transmit various information from the therapy assessment module (108) to the therapist computing device (102a) for the therapist to review after the psychotherapy session has ended.
  • the information to be transmitted and reviewed may comprise the first indication, second indication, third indication, fourth indication, first result, second result, third result and fourth result.
  • the text-based speech data and information may be transmitted from the NLP module (104) and the therapist assessment module (108) to the therapist computing device (102a) simultaneously.
  • the feedback module (130) may enable the therapist to review the indications and results received from the therapy assessment module (110), upon displaying via the user interface on the display device of the therapist computing device (102a).
  • the feedback module (130) may enable the therapist to provide a confirmation input through the user interface on the therapist computing device (102a).
  • the feedback module (130) may be configured to generate a signal to the therapy assessment module (110), notifying that the therapy assessment module (110) may upload all of the information to the database (120).
  • the feedback module (130) may enable the therapist to provide a comment input comprising comments on the detected suspicious trends through the user interface on the therapist computing device (102a). Subsequently, the feedback module (130) may be configured to convert the comment input to a plurality of comment data, so that the comment data can be tagged to the corresponding indications or results, before uploading all of the information to the database (120). In certain embodiments, the feedback module (130) may also be configured to transfer the comment data to the therapy assessment module (110), so that the tagging may be performed by the therapy assessment module (110).
  • NLP module (104), paralinguistic feature analysing module (106), baseline state analysing module (108), therapy assessment module (110), database (120) and feedback module (130) may each be a separate or individual module, it should be appreciated that in certain embodiments, these modules may be integral parts of the therapist computing device (102a), depending on the system configuration.
  • first indication, second indication, third indication, fourth indication, baseline indication, first result, second result, third result and fourth result may be expressed in term of scores. Accordingly, these scores may be provided and compared with ranges comprising a low range, a normal range and a high range. In some embodiments, these ranges may also be updated periodically.
  • Figure 2 shows a system for conducting a psychotherapy session remotely and over the internet between a therapist and more than one client, according to another embodiment of the invention.
  • Each computing device (102a, 102b) may also be configured to capture and record the reactions of the therapist and each of the clients during the psychotherapy session using the video camera and microphone provided.
  • the reactions to be captured and recorded may comprise one or more of the following: the therapist’s speeches, the client’s speeches, the therapist’s body paralinguistic features (such as the therapist’s facial expressions, eye movements, gestures and postures), the client’s body paralinguistic features (such as the client’s facial expressions, eye movements, gestures and postures), the therapist’s vocal paralinguistic features (such as the therapist’s pitch, loudness or sound pressure, timbre and tone) and the client’s vocal paralinguistic features (such as the client’s pitch, loudness or sound pressure, timbre and tone).
  • the reactions may be recorded in the form of either an audio file or a video file.
  • the speeches and vocal paralinguistic features of Client A2 may be recorded in the form of an audio file
  • the body paralinguistic features of Client A2 may be recorded in the form of a video file.
  • the audio and video files may be processed to provide the therapist with insightful and helpful indications in assessing the psychotherapy session including the therapeutic relationship or alliance between the therapist and each of the clients.
  • the system may be provided with several processing modules, such as a natural language processing (NLP) module (104) and a paralinguistic analysing module (106).
  • NLP natural language processing
  • the system may also be configured to enable data transmission or communication between the computing devices (102a, 102b), NLP module (104) and paralinguistic analysing module (106).
  • the NLP module (104) may be configured to receive the audio files from the therapist computing device (102a) and each of the client computing devices (102b) in real-time or near real-time during a psychotherapy session.
  • the NLP module (104) may also be configured to convert the speeches in the audio files to text-based speech data and then analyse the speech data to generate at least a first indication and a second indication for each of the clients participated in the psychotherapy session.
  • the first indication may be associated with a degree of verbal concordance between the therapist and a particular client, whereas the second indication may be associated with an insight data in response to a particular therapist-client scenario.
  • the NLP module (104) may also be configured to analyse the text-based speech data for generating a fifth indication, in which the fifth indication may be associated with a degree of verbal concordance between two clients, such as between Client A2 and Client B2, between Client A2 and Client C2, between Client B2 and Client C2, etc.
  • the paralinguistic feature analysing module (106) may be configured to receive the audio files comprising the vocal paralinguistic features and video files comprising the body paralinguistic features from the therapist computing device (102a) and client computing devices (102b). Similarly, the transfer of the audio and video files from the computing devices (102a, 102b) to the paralinguistic feature analysing module (106) may take place in real-time or near real-time during the psychotherapy session. Upon receipt of the audio and video files, the paralinguistic feature analysing module (106) may be configured to convert the paralinguistic features to a plurality of paralinguistic data and then analyse the paralinguistic data to generate at least a third indication for each of the clients participated in the psychotherapy session.
  • the third indication may be associated with a psychological state of a particular client, wherein the psychological state may be a mental state (such as fatigue, sleepiness, concentration, etc.), emotional state (such as anger, fear, sadness, frustration, etc.), socio-emotional state (such as guilt, embarrassmentjealousy, etc.) or any combination thereof.
  • the psychological state may preferably be the concentration level of a particular client.
  • the paralinguistic feature analysing module (106) may also analyse the paralinguistic data to generate a fourth indication and a sixth indication for each of the clients in the psychotherapy session.
  • the fourth indication may be associated with a degree of nonverbal concordance between the therapist and a particular client
  • the sixth indication may be associated with a degree of non-verbal concordance between two of the clients in the psychotherapy session, such as between Client A2 and Client B2, between Client A2 and Client C2, between Client B2 and Client C2, etc.
  • the paralinguistic feature analysing module (106) may be made up of a body paralinguistic feature analysing submodule (to analyse only body paralinguistic features) and a vocal paralinguistic feature analysing sub-module (to analyse only vocal paralinguistic features), the sub-modules being capable of establishing a data communication with each other.
  • the system may also comprise a baseline state analysing module (108) configured to be in data communication with the NLP module (104), paralinguistic feature analysing module (106) and client computing devices (102b).
  • the baseline state analysing module (108) may additionally be configured to determine a baseline indication associated with the baseline state or initial state for each client, at the beginning of the psychotherapy session. To determine the baseline indication, the baseline state analysing module (108) may be configured to prompt and invite each of the clients to react to a list of questions displayed to the client through the client computing devices (102b) or posed to the client by the therapist at the beginning of the psychotherapy session.
  • the reactions in response to the questions may be captured and recorded by the respective client computing devices (102b), wherein the reactions to be captured and recorded may comprise the client’s speeches, the client’s body paralinguistic features (such as the client’ s facial expressions, eye movements, gestures and postures) and the client’ s vocal paralinguistic features (such as the client’s pitch, loudness or sound pressure, timbre and tone).
  • the reactions of the clients may each be analysed by the NLP module (104) and paralinguistic feature analysing module (106). Based on the analyses performed by the NLP module (104) and paralinguistic feature analysing module (106), the baseline state analysing module (108) could determine a baseline indication associated with the baseline state for each of the clients.
  • the baseline indications may further be processed to assess fluctuations in the emotional valence of a particular client (or the emotional valence of each client) during the psychotherapy session. If more than a psychotherapy session may be conducted, the baseline indication for each psychotherapy session may be recorded and accumulated, hence allowing for monitoring of the overall progress of the same client (or each client engaged in the psychotherapy session) and effectiveness of the therapeutic treatment plan. Accumulation of the baseline indications may also enable changes in the client’s psychological or emotional states to be tracked within the present psychotherapy session and across multiple psychotherapy sessions.
  • multiple first indications, second indications, third indications, fourth indications, fifth indications, sixth indications and baseline indications may be generated and obtained, as each client may have his first indication, second indication, third indication, fourth indication, fifth indication, sixth indication and baseline indication.
  • the system may also comprise a therapy assessment module (110) configured to be in data communication with the NLP module (104), the paralinguistic feature analysing module (106) and the baseline state analysing module (108), in order to receive the first indications, second indications, third indications, fourth indications, fifth indications, sixth indications and baseline indications therefrom.
  • the therapy assessment module (110) may also be configured to assess the psychotherapy session upon processing these indications.
  • the therapy assessment module (110) may be configured to determine one or more of the following: a first result in relation to the therapeutic relationship or alliance between the therapist and the client, based on the first and fourth indications; a second result in relation to the client’s emotional valence score, based on the second, third and baseline indications; a third result related to whether the client is honest or dishonest, based on the second and third indications; a fourth result related to whether the therapeutic approach is effective, based on the second indication; and a fifth result in relation to the therapeutic relationship between the client and another client, based on the fifth and sixth indications.
  • the system may also comprise a database (120).
  • the database (120) may be configured to be in data communication with the NLP module (104), the paralinguistic feature analysing module (106), the baseline state analysing module (108) and the therapy assessment module (110) respectively.
  • the database (120) may also be configured to store a plurality of data, for instance but not limited to the audio files, video files, text-based speech data, body paralinguistic data, vocal paralinguistic data, indications and results generated.
  • the data may be sorted and stored in the database (120) according to the client’s name. For instance, all data pertaining to a psychotherapy session between Therapist A2, Client A2 and Client B2 may be duplicated, so that the data may be stored under a record belonging to Client A2 and a record belonging to Client B2 respectively.
  • the record of Client A2 or Client B2 may comprise one or more of the following: Therapist A2’s audio files, Therapist A2’S video files, Client A2’s audio files, Client A2’s video files, Client B2’s audio files, Client B2’s video files, Therapist A2’s speech data, Client A2’s speech data, Client B2 S speech data, Therapist A2’s body paralinguistic data, Client A2’s body paralinguistic data, Client B2’s body paralinguistic data, Therapist A2’s vocal paralinguistic data, Client A2’s vocal paralinguistic data, Client B2’s vocal paralinguistic data, indications and results generated.
  • more than one database (120) may be provided depending on the system configuration.
  • the system may further comprise a feedback module (130) in data communication with the NLP module (104) and the therapy assessment module (110).
  • a feedback module (130) in data communication with the NLP module (104) and the therapy assessment module (110).
  • presence of the feedback module (130) may enable a feedback loop to be established, hence allowing continuous training, retraining or fine-tuning to the steps, operations or algorithms to be executed. Consequently, the accuracy of the analyses and results could be improved.
  • the feedback module (130) may be configured to mitigate issues caused by the recorded speeches which are of poor quality or unclear.
  • the NLP module (104) may encounter difficulties in identifying the correct words based on the recorded speeches.
  • the text-based speech data produced by the NLP module (104) may be erroneous and inaccurate, thereby affecting the assessment of the psychotherapy session or in particular the performance of the therapy assessment module (110).
  • the feedback module (130) may be configured to transfer and display the text-based speech data to the therapist for viewing via a user interface on a display device of the therapist computing device (102a) after the psychotherapy session has ended.
  • the user interface may also allow the therapist to provide a confirmation input that the text-based speech data do not require corrections.
  • the feedback module (130) may be configured to transmit a signal to the NLP module (104), notifying that the NLP module (104) may upload the originally generated speech data to the database (120).
  • the feedback module (130) may enable the therapist to provide a correction input through the user interface mentioned in the foregoing, to correct the errors in the speech data. Subsequently, the feedback module (130) may transfer the corrected speech data to the NLP module (104), so that the NLP module (104) may generate the updated indications based on the corrected speech data, before transferring the updated indications to the therapy assessment module (110) to update the results and/or before uploading the latest information (e.g. the corrected speech data, updated indications and updated results) to the database (120).
  • the latest information e.g. the corrected speech data, updated indications and updated results
  • the feedback module (130) may be further configured to transmit various information from the therapy assessment module (108) to the therapist computing device (102a) for the therapist to review, after the psychotherapy session has ended.
  • the information to be transmitted and reviewed may comprise the first indications, second indications, third indications, fourth indications, fifth indications, sixth indications, first results, second results, third results, fourth results and fifth results.
  • the text-based speech data and information may be transmitted simultaneously from the NLP module (104) and the therapy assessment module (110) to the therapist computing device (102a).
  • the feedback module (130) may enable the therapist to review the indications and results received from the therapy assessment module (110), upon displaying via the user interface on the display device of the therapist computing device (102a).
  • the feedback module (130) may enable the therapist to provide a confirmation input through the user interface on the therapist computing device (102a). Upon receipt of the confirmation input, the feedback module (130) may be configured to generate a signal to the therapy assessment module (110), notifying that the therapy assessment module (110) may upload all of the information to the database (120).
  • the feedback module (130) may enable the therapist to provide a comment input comprising comments on the detected suspicious trends through the user interface on the therapist computing device (102a). Subsequently, the feedback module (130) may be configured to convert the comment input to a plurality of comment data, so that the comment data can be tagged to the corresponding indications or results, before uploading all of the information to the database (120). In alternative embodiments, the feedback module (130) may be configured to transfer the comment data to the therapy assessment module (110), so that the tagging may be performed by the therapy assessment module (110).
  • NLP module (104), paralinguistic feature analysing module (106), baseline state analysing module (108), therapy assessment module (110), database (120) and feedback module (130) may each be a separate or individual module, it should be appreciated that in some embodiments, these modules may be integral parts of the therapist computing device (102a), depending on the system configuration.
  • first indications, second indications, third indications, fourth indications, fifth indications, sixth indications, baseline indications, first results, second result, third result, fourth results and fifth results may be expressed in terms of scores. Accordingly, these scores may be provided and compared with ranges comprising a low range, a normal range and a high range. In some embodiments, these ranges may be updated periodically.
  • a method for conducting a remote psychotherapy between a therapist and a client may be provided. The method may also be performed by using the system described in the foregoing.
  • the client or the patient may be required to schedule an appointment with the therapist, as at Step S201 in Figure 3. Later, at the appointed date and time, the therapist and the client may be required to establish a communication session, as at Step S203, via a web application or web browsing application accessible on the therapist computing device (102a) and the client computing client (102b). To establish such communication session, the computing devices (102a, 102b) may be required to connect to the internet first.
  • the psychotherapy conducted between the therapist and client may also be preferably a two-way audio-video communication session.
  • the therapist computing device (102a) and the client computing device (102b) may respectively be provided with at least a video camera to record an image of a person, a microphone to record sound of a person and a speaker to play audio to a person.
  • a baseline indication associated with the client may be required to determine a baseline indication associated with the client’s baseline state (or “initial state”) at the beginning of the psychotherapy session.
  • the client may be invited and required by the baseline analysing module (108), at Step S205, to react to a plurality of questions displayed to the client on the client computing device (102b) or posed to the client by the therapist at the beginning of the psychotherapy session.
  • the client’s reactions in response to the questions may be captured and recorded using the video camera and microphone provided to the client computing device (102b), wherein the reactions to be captured and recorded may comprise the client’s speeches, the client’s body paralinguistic features (such as the client’s facial expressions, eye movements, gestures and postures) and the client’s vocal paralinguistic feature (such as the client’s pitch, loudness or sound pressure, timbre and tone).
  • the client’s reactions may also be analysed using the NLP module (104) and paralinguistic feature analysing module (106). Subsequently, based on the analyses performed by the NLP module (104) and paralinguistic feature analysing module (106), a baseline score associated with the client’s baseline state may be computed by the baseline analysing module (108), at Step S207.
  • the psychotherapy may be initiated, as at Step S209.
  • the therapist’ s reactions and client’s reactions may be captured and recorded, as at Step S211 and Step S221, using the video camera and microphone provided to the computing devices (102a, 102b).
  • the reactions to be captured and recorded may be one or more of the following: the therapist’s speeches, the client’s speeches, the therapist’s body paralinguistic features (such as the therapist’s facial expressions, eye movements, gestures and postures), the client’s body paralinguistic features (such as the client’s facial expressions, eye movements, gestures and postures), the therapist’s vocal paralinguistic features (such as the therapist’s pitch, loudness or sound pressure, timbre and tone) and the client’s vocal paralinguistic features (such as the client’s pitch, loudness or sound pressure, timbre and tone).
  • the reactions may be recorded in the form of an audio file or a video file.
  • the reactions captured and recorded are the therapist’s and the client’s speeches
  • the reactions may be recorded in the form of an audio file.
  • These audio files may then be transferred to the NLP module (104) which converts the speeches in the audio files to text-based speech data, as at Step S213.
  • the NLP module (104) may analyse the speech data to generate at least a first indication and a second indication, wherein the first indication may be associated with a degree of verbal concordance between the therapist and the client, and the second indication may be associated with an insight data in response to a particular therapist-client scenario.
  • the NLP module (104) may compare the therapist’s speech data and the client’s speech data by using a Language Style Matching (LSM) algorithm to identify one or more function words from different categories.
  • the function word categories may include but not limited to auxiliary verbs, articles, common adverbs, particle pronouns, indefinite impersonal pronouns, prepositions and relative pronouns, negations, conjunctions and quantifiers.
  • a score may be calculated by the NLP module (104) for each of the function word categories by using Equation (1), wherein FW is the number of function words. Accordingly, by taking an average value of the scores calculated for each function word category, a composite LSM score may be obtained and referred to as the “first indication”. It also indicates great matching between the therapist and client if the composite LSM score is high.
  • the NLP module (104) may analyse the therapist’s speech data and the client’s speech data to identify certain keywords and compare the identified keywords with a list of insight conditions, before generating a second indication associated with an insight data in response to a particular therapist-client scenario.
  • the NLP module (104) may analyse the client’s speech data and identify or detect that the client is using a high rate of emotion words during the psychotherapy session. Hence, the NLP module (104) may generate a second indication alerting the therapist of such situation.
  • the second indication may also comprise a suggestion to the therapist, such as minimising the use of cognitively geared verbs in the communication between the therapist and the client.
  • the NLP module (104) may analyse the therapist’s speech data and identify or detect that the therapist’s speeches do not contain words or phrases characterising a therapeutic approach adopted by the therapist. Accordingly, the NLP module (104) may generate a second indication alerting the therapist of such situation. If the reactions captured and recorded are the paralinguistic features of the therapist and the paralinguistic features of the client, the reactions may be recorded in the form of a video file. These video files may be transferred to the paralinguistic feature analysing module (106) which may convert the paralinguistic features contained in the video files to a plurality of paralinguistic data, as at Step S223, and analyse the paralinguistic data to generate at least a third indication, as at Step S225.
  • the paralinguistic feature analysing module (106) may process the client’s body paralinguistic features by using a facial action recognition algorithm which identifies and analyses movements of different facial muscles. Additionally, the paralinguistic feature analysing module (106) may process the vocal paralinguistic features of the client by identifying and analysing different vocal characteristics. Based upon the analyses on the client’s body and vocal paralinguistic features, the paralinguistic feature analysing module (106) may generate the third indication associated with a psychological state of the client.
  • the psychological state may be a mental state (such as fatigue, sleepiness, concentration, etc.), emotional state (such as anger, fear, sadness, frustration, etc.) or a combination of both.
  • the psychological state may preferably be the client’s concentration level.
  • the paralinguistic feature analysing module (106) may also analyse the paralinguistic data to generate a fourth indication associated with a degree of non-verbal concordance between the therapist and the client.
  • first indication, second indication, third indication, fourth indication and baseline indication may each relate to a specific aspect of the psychotherapy session, these indications may be subjected to further processing using the therapy assessment module (110), in order to enable more meaningful and insightful assessment in relation to the psychotherapy session.
  • the therapy assessment module (110) may determine one or more of the following: a first result in relation to the therapeutic relationship or alliance between the therapist and client, based on the first and fourth indications; a second result in relation to the client’s emotional valence score, based the second, third and baseline indications; a third result related to whether the client is honest or dishonest, based on the second and third indications; and a fourth result related to whether the therapeutic approach is effective, based on the second indication.
  • various information may be transmitted from the therapy assessment module (110) to the therapist computing device (102a) through the feedback module (130) after the psychotherapy session has ended, as at Step S241 in Figure 4, thereby enabling the therapist to review the information before uploading and storing them onto a database (120), as at Step S245 in Figure 4.
  • the information to be reviewed may comprise the first indication, second indication, third indication, fourth indication, first result, second result and third result.
  • the feedback module (130) may transfer and display the indications and results to the therapist for viewing through a user interface on a display device of the therapist computing device (102a). Accordingly, it enables the therapist to review and provide feedbacks about the indications and results. During review, if the therapist does not detect any suspicious trends or have comments on the indications and results, the therapist may provide a confirmation input to the feedback module (130) via the user interface on the therapist computing device (102a). Upon the confirmation input, the feedback module (130) may generate a signal to the therapy assessment module (110), as at Step S251, notifying that the therapy assessment module (110) may upload all of the information to the database (120), as at Step S253.
  • the therapist may provide a comment input comprising comments on the detected suspicious trends to the feedback module (130) through the user interface on the therapist computing device (102a). Subsequently, the feedback module (130) may convert the comment input to a plurality of comment data, so that they can be tagged to the corresponding indications or results, before uploading all of the information to the database (120).
  • the text-based speech data may also be transmitted from the NLP module (104) to the therapist computing device (102a) for review, via the feedback module (130), as at Step S243 and Step S245 in Figure 4.
  • the feedback module (130) may transfer and display the text-based speech data to the therapist for viewing through the user interface on the therapist computing device (102a).
  • the therapist may provide a correction input through the user interface, in order to correct the errors in the speech data.
  • the feedback module (130) may then transfer the corrected speech data back to the NLP module (104), as at Step S251, thereby enabling the NLP module (104) to update the first and second indications based on the corrected speech data, before transferring the updated indications to the therapy assessment module (108) to update the results and/or uploading the latest information (e.g. the corrected speech data, updated indications and updated results) to the database (120), as at Step S253.
  • the therapist may provide a confirmation input through the user interface on the therapist computing device (102a) to the feedback module (130).
  • the feedback module (130) may generate and transmit a signal to the NLP module (104), as at Step S251, notifying that the NLP module (104) may upload the originally generated speech data to the database (120), as at Step S253.
  • first indication, second indication, third indication, fourth indication, baseline indication, first result, second result, third result and fourth result may be expressed in term of scores. Accordingly, these scores may be provided and compared with ranges comprising a low range, a normal range and a high range. In some further embodiments, these ranges may be updated periodically.
  • a method for conducting a remote psychotherapy session between a therapist and more than a client may be provided. The method may also be performed using the system described in the foregoing.
  • two or more clients may be required to schedule an appointment with the therapist, as at Step S301 in Figure 5.
  • the therapist and the clients may be required to establish a communication session, as at Step S303, via a web application or a web browsing application accessible on the therapist computing device (102a) and the client computing devices (102b).
  • the computing devices (102a, 102b) may be required to connect to the internet first.
  • the psychotherapy conducted between the therapist and several clients may preferably be a two-way audio-video communication session.
  • the therapist computing device (102a) and the client computing devices (102b) may respectively be provided with at least a video camera to record an image of a person, a microphone to record sound of a person and a speaker to play audio to a person.
  • each of the clients may be invited and required by the baseline analysing module (108), at Step S305, to react to a plurality of questions displayed to the client on the client computing device (102b) or posed to the client by the therapist at the beginning of the psychotherapy session.
  • the client’s reactions in response to the questions may be captured and recorded by using the video camera and microphone provided to the client computing device (102b).
  • the reactions to be captured and recorded may comprise the client’s speeches, the client’s body paralinguistic features (such as the client’s facial expressions, eye movements, gestures and postures) and the client’s vocal paralinguistic feature (such as the client’s pitch, loudness or sound pressure, timbre and tone).
  • the client’s reactions may also be analysed by using the NLP module (104) and paralinguistic feature analysing module (106). Subsequently, based on the analyses performed by the NLP module (104) and the paralinguistic feature analysing module (106), a baseline score associated with the baseline state may be computed for each of the clients by the baseline analysing module (108), at Step S307. Once the baseline indications are determined for each of the clients, the psychotherapy may be initiated, as at Step S309.
  • the reactions of the therapist and each of the clients may be captured and recorded, as at Step S311 and Step S321, using the video camera and microphone provided to the therapist computing device (102a) and the client computing devices (102b).
  • the reactions to be captured and recorded may be one or more of the following: the therapist’s speeches, the client’s speeches, the therapist’s body paralinguistic features (such as the therapist’s facial expressions, eye movements, gestures and postures), the client’s body paralinguistic features (such as the client’s facial expressions, eye movements, gestures and postures), the therapist’s vocal paralinguistic features (such as the therapist’s pitch, loudness or sound pressure, timbre and tone) and the client’s vocal paralinguistic features (such as the client’s pitch, loudness or sound pressure, timbre and tone).
  • the reactions may be recorded in the form of an audio file or a video file.
  • the reactions captured and recorded are the therapist’s and the client’s speeches, the reactions may be recorded in the form of an audio file.
  • These audio files may then be transferred to the NLP module (104) which converts the speeches in the audio files to text-based speech data, as at Step S313.
  • the NLP module (104) may analyse the speech data to generate at least a first indication and a second indication for each of the clients participated in the psychotherapy session, wherein the first indication may be associated with a degree of verbal concordance between the therapist and a particular client, and the second indication may be associated with an insight data in response to a specific therapist-client scenario.
  • the NLP module (104) may also analyse the speech data to generate a fifth indication which may be associated with a degree of verbal concordance between the clients, such as between Client A4 and B4, between Client A4 and Client C4, between Client B4 and Client C4, etc.
  • first, second and fifth indications may be generated, as more than one client is participated in the psychotherapy session.
  • the same operations depicted in Embodiment 3 above may be performed for obtaining the first and second indications for each of the clients participated in the psychotherapy session, while the fifth indications may be calculated and obtained using the same operations for obtaining the first indications.
  • the reactions captured and recorded are the body and vocal paralinguistic features of each of the therapist and clients, the reactions may be recorded in the form of a video file.
  • These video files may be transferred to the paralinguistic feature analysing module (106) which may convert the paralinguistic features contained in the video files to a plurality of paralinguistic data, as at Step S323, and analyse the paralinguistic data to generate at least a third indication for each of the clients in the psychotherapy session, as at Step S325.
  • the third indication may be associated with a psychological state of a particular client, wherein the psychological state may be a mental state (such as fatigue, sleepiness, concentration, etc.), emotional state (such as anger, fear, sadness, frustration, etc.), socio-emotional state (such as guilt, embarrassment, ashamedy, etc.) or any combination thereof.
  • the psychological state may preferably be the concentration level of a particular client.
  • the paralinguistic feature analysing module (106) may also analyse the paralinguistic data to generate a fourth indication and a sixth indication for each client in the psychotherapy session.
  • the fourth indication may be associated with a degree of non-verbal concordance between the therapist and a particular client
  • the sixth indication may be associated with a degree of non-verbal concordance between a pair of clients participated in the psychotherapy session, such as between Client A4 and B4, between Client A4 and Client C4, between Client B4 and Client C4, etc.
  • third, fourth and sixth indications may be generated, as more than one client is participated in the psychotherapy session.
  • the same operations described in Embodiment 3 above may also be performed to generate the third indication for each of the clients participated in the psychotherapy session.
  • the first, second, third, fourth, fifth, sixth and baseline indications may be subjected to further processing using the therapy assessment module (110), in order to enable more meaningful and insightful assessment in relation to the psychotherapy session.
  • the therapy assessment module (110) may determine one or more of the following for each client in the psychotherapy session: a first result in relation to the therapeutic relationship or alliance between the therapist and the client, based on the first and fourth indications; a second result in relation to the client’s emotional valence score, based the second, third and baseline indications; a third result related to whether the client is honest or dishonest, based on the second and third indications; a fourth result related to whether the therapeutic approach is effective, based on the second indication; and a fifth result in relation to the therapeutic relationship between the client and another client, based on the fifth and sixth indications.
  • various information may be transmitted from the therapy assessment module (110) via the feedback module (130) to the therapist computing device (102a), as at Step S341 in Figure 6. It may then enable the therapist to review the information, as at Step 345, prior to uploading and storing them onto a database (120).
  • the information to be reviewed at Step S345 may comprise the first indications, second indications, third indications, fourth indications, fifth indications, sixth indications, first results, second results, third results and fourth results.
  • the feedback module (130) may transfer and display the indications and results to the therapist for viewing through a user interface on a display device of the therapist computing device (102a). Accordingly, it enables the therapist to review and provide feedbacks about the indications and results. During review, if the therapist does not detect any suspicious trends or have comments on the indications and results, the therapist may be requested to provide a confirmation input to the feedback module (130) through the user interface on the therapist computing device (102a). Upon the confirmation input, the feedback module (130) may generate a signal to the therapy assessment module (110), as at Step S351, notifying that the therapy assessment module (110) may upload all of the information to the database (120), as at Step S353.
  • the therapist may provide a comment input comprising the therapist’s comments on the detected suspicious trends to the feedback module (130) via the user interface on the therapist computing device (102a). Subsequently, the feedback module (130) may convert the comment input to a plurality of comment data, so that they can be tagged to the corresponding indications or results, before uploading all of the information to the database (120).
  • the text-based speech data may also be transmitted from the NLP module (104) to the therapist computing device (102a) for review, via the feedback module (130), as at Steps S343 and S345 in Figure 6.
  • the feedback module (130) may transfer and display the text-based speech data to the therapist for viewing through the user interface on the therapist computing device (102a). If the therapist identifies or detects errors in the text-based speech data, the therapist may provide a correction input through the user interface to correct the errors in the speech data. Later, the feedback module (130) may transfer the corrected speech data back to the NLP module (104), as at Step S351, thereby enabling the NLP module (104) to update the first, second and fifth indications based on the corrected speech data, before transferring the indications to the therapy assessment module (108) to update the results and/or before uploading the latest information (e.g. the corrected speech data, updated indications and updated results) to the database (120), as at Step S353.
  • the latest information e.g. the corrected speech data, updated indications and updated results
  • the therapist may provide a confirmation input via the user interface on the therapist computing device (102a) to the feedback module (130).
  • the feedback module (130) may generate and transmit a signal to the NLP module (104), as at Step S351, notifying that the NLP module (104) may upload the originally generated data to the database (120), as at Step S353.
  • first indications, second indications, third indications, fourth indications, fifth indications, sixth indications, baseline indications, first results, second results, third results and fourth results may be expressed in term of scores. Accordingly, these scores may be provided and compared with ranges comprising a low range, a normal range and a high range. In some further embodiments, these ranges may be updated periodically.
  • the methods described in the preceding description may be converted to a series of computer-executable program instructions stored on a non-transitory computer-readable storage medium.
  • the program instructions When the program instructions are executed by either a computing device or a processing module of the computing device, it may cause the processing module to perform the steps or operation outlined above.
  • a computer program product comprising a computer useable or readable medium having a computer readable program may be provided.
  • the computer readable program when executed on a computing device, may cause the computing device to perform the steps or operations outlined above.

Landscapes

  • Health & Medical Sciences (AREA)
  • Engineering & Computer Science (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Physics & Mathematics (AREA)
  • Public Health (AREA)
  • Medical Informatics (AREA)
  • Psychiatry (AREA)
  • Pathology (AREA)
  • Biomedical Technology (AREA)
  • Psychology (AREA)
  • Hospice & Palliative Care (AREA)
  • Social Psychology (AREA)
  • Child & Adolescent Psychology (AREA)
  • Developmental Disabilities (AREA)
  • Epidemiology (AREA)
  • Multimedia (AREA)
  • Human Computer Interaction (AREA)
  • Veterinary Medicine (AREA)
  • Animal Behavior & Ethology (AREA)
  • Biophysics (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Molecular Biology (AREA)
  • Surgery (AREA)
  • Primary Health Care (AREA)
  • Theoretical Computer Science (AREA)
  • Educational Technology (AREA)
  • General Physics & Mathematics (AREA)
  • Computational Linguistics (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Signal Processing (AREA)
  • Artificial Intelligence (AREA)
  • Acoustics & Sound (AREA)
  • Data Mining & Analysis (AREA)
  • Databases & Information Systems (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • Fuzzy Systems (AREA)
  • Evolutionary Computation (AREA)
  • General Engineering & Computer Science (AREA)

Abstract

L'invention concerne un système pour effectuer une session de psychothérapie à distance entre un thérapeute équipé d'un dispositif informatique de thérapeute (102a) et au moins un client équipé d'un dispositif informatique client (102b). Dans un mode de réalisation, le système peut comprendre un module de traitement de langage naturel (NLP) (104) et un module d'analyse de caractéristique paralinguistique (106) pour convertir et analyser les réactions du thérapeute et les réactions du client ou des clients capturées pendant la session de psychothérapie, générant ainsi des indications qui sont utiles et judicieuses dans l'évaluation de la session de psychothérapie. L'invention concerne également un procédé pour sa réalisation.
PCT/MY2022/050047 2021-08-09 2022-06-15 Systèmes et procédés pour conduire et évaluer des sessions de psychothérapie à distance WO2023018325A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
MYPI2021004588 2021-08-09
MYPI2021004588 2021-08-09

Publications (1)

Publication Number Publication Date
WO2023018325A1 true WO2023018325A1 (fr) 2023-02-16

Family

ID=85200878

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/MY2022/050047 WO2023018325A1 (fr) 2021-08-09 2022-06-15 Systèmes et procédés pour conduire et évaluer des sessions de psychothérapie à distance

Country Status (1)

Country Link
WO (1) WO2023018325A1 (fr)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2004030532A1 (fr) * 2002-10-03 2004-04-15 The University Of Queensland Methode et appareil d'evaluation des troubles psychiatriques ou physiques
WO2019246239A1 (fr) * 2018-06-19 2019-12-26 Ellipsis Health, Inc. Systèmes et procédés d'évaluation de santé mentale
US20210090733A1 (en) * 2019-09-19 2021-03-25 International Business Machines Corporation Automatic detection of mental health condition and patient classification using machine learning
WO2021081418A1 (fr) * 2019-10-25 2021-04-29 Ellipsis Health, Inc. Modèles de traitement de langage acoustique et naturel pour la sélection par commande vocale et la surveillance de conditions de santé comportementale
US11004461B2 (en) * 2017-09-01 2021-05-11 Newton Howard Real-time vocal features extraction for automated emotional or mental state assessment

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2004030532A1 (fr) * 2002-10-03 2004-04-15 The University Of Queensland Methode et appareil d'evaluation des troubles psychiatriques ou physiques
US11004461B2 (en) * 2017-09-01 2021-05-11 Newton Howard Real-time vocal features extraction for automated emotional or mental state assessment
WO2019246239A1 (fr) * 2018-06-19 2019-12-26 Ellipsis Health, Inc. Systèmes et procédés d'évaluation de santé mentale
US20210090733A1 (en) * 2019-09-19 2021-03-25 International Business Machines Corporation Automatic detection of mental health condition and patient classification using machine learning
WO2021081418A1 (fr) * 2019-10-25 2021-04-29 Ellipsis Health, Inc. Modèles de traitement de langage acoustique et naturel pour la sélection par commande vocale et la surveillance de conditions de santé comportementale

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
KOOLE SANDER L., TSCHACHER WOLFGANG: "Synchrony in Psychotherapy: A Review and an Integrative Framework for the Therapeutic Alliance", FRONTIERS IN PSYCHOLOGY, vol. 7, XP093035755, DOI: 10.3389/fpsyg.2016.00862 *
SIKORSKI, WIESLAW: "Paralinguistic communication in the therapeutic relationship", ARCHIVES OF PSYCHIATRY & PSYCHOTHERAPY, vol. 14, no. 1, 30 November 2011 (2011-11-30), pages 49 - 54, XP009543611, ISSN: 1509-2046 *

Similar Documents

Publication Publication Date Title
US11942194B2 (en) Systems and methods for mental health assessment
US11120895B2 (en) Systems and methods for mental health assessment
Luo et al. A critical review of state‐of‐the‐art chatbot designs and applications
CN106685916B (zh) 电子会议智能装置及方法
US11551804B2 (en) Assisting psychological cure in automated chatting
CN106686339B (zh) 电子会议智能
US20220328064A1 (en) Acoustic and natural language processing models for speech-based screening and monitoring of behavioral health conditions
US20190243916A1 (en) Cognitive Ranking of Terms Used During a Conversation
US20180018966A1 (en) System for understanding health-related communications between patients and providers
US10885080B2 (en) Cognitive ranking of terms used during a conversation
CN110377719B (zh) 医疗问答方法及装置
US11616740B2 (en) Invoking chatbot in online communication session
US11907820B2 (en) Automated bias elimination in negotiated terms
US11862302B2 (en) Automated transcription and documentation of tele-health encounters
Baker et al. Approaches for assessing communication in human-autonomy teams
JP2021533489A (ja) フィードバックを収集するためのコンピュータ実装システム及び方法
WO2023018325A1 (fr) Systèmes et procédés pour conduire et évaluer des sessions de psychothérapie à distance
US20230062127A1 (en) Method for collaborative knowledge base development
US20220093220A1 (en) System and method for patient assessment using disparate data sources and data-informed clinician guidance via a shared patient/clinician user interface
Liu et al. Stone needle: A general multimodal large-scale model framework towards healthcare
Wang et al. Study on the classification problem of the coping stances in the Satir model based on machine learning
EP4102812A1 (fr) Système de communication électronique et procédé utilisant des informations des événements biométriques
US11741298B1 (en) Real-time meeting notes within a communication platform
US20230036171A1 (en) Systems and methods for rapid vetting of counselors via graded simulated exchanges
Hasan et al. Thesis Proposal: Detecting Empathy Using Multimodal Language Model

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22856315

Country of ref document: EP

Kind code of ref document: A1

DPE1 Request for preliminary examination filed after expiration of 19th month from priority date (pct application filed from 20040101)
WWE Wipo information: entry into national phase

Ref document number: 11202400165S

Country of ref document: SG

NENP Non-entry into the national phase

Ref country code: DE