WO2021261887A1 - Appareil et procédé d'analyse d'état mental utilisant un agenda vidéo - Google Patents

Appareil et procédé d'analyse d'état mental utilisant un agenda vidéo Download PDF

Info

Publication number
WO2021261887A1
WO2021261887A1 PCT/KR2021/007826 KR2021007826W WO2021261887A1 WO 2021261887 A1 WO2021261887 A1 WO 2021261887A1 KR 2021007826 W KR2021007826 W KR 2021007826W WO 2021261887 A1 WO2021261887 A1 WO 2021261887A1
Authority
WO
WIPO (PCT)
Prior art keywords
diary
psychological state
video
state analysis
psychological
Prior art date
Application number
PCT/KR2021/007826
Other languages
English (en)
Korean (ko)
Inventor
윤순일
정윤진
Original Assignee
주식회사 플랙스
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 주식회사 플랙스 filed Critical 주식회사 플랙스
Publication of WO2021261887A1 publication Critical patent/WO2021261887A1/fr

Links

Images

Classifications

    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/16—Devices for psychotechnics; Testing reaction times ; Devices for evaluating the psychological state
    • A61B5/165—Evaluating the state of mind, e.g. depression, anxiety
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/48—Other medical applications
    • A61B5/4803—Speech analysis specially adapted for diagnostic purposes
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/74—Details of notification to user or communication with user or patient ; user input means
    • A61B5/742—Details of notification to user or communication with user or patient ; user input means using visual displays
    • A61B5/744—Displaying an avatar, e.g. an animated cartoon character
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/74—Details of notification to user or communication with user or patient ; user input means
    • A61B5/7465—Arrangements for interactive communication between patient and care services, e.g. by using a telephone network
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00—Speech recognition
    • G10L15/08—Speech classification or search
    • G10L15/18—Speech classification or search using natural language modelling
    • G10L15/183—Speech classification or search using natural language modelling using context dependencies, e.g. language models
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L25/00—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
    • G10L25/03—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 characterised by the type of extracted parameters
    • G10L25/15—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 characterised by the type of extracted parameters the extracted parameters being formant information
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L25/00—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
    • G10L25/48—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use
    • G10L25/51—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use for comparison or discrimination
    • G10L25/63—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use for comparison or discrimination for estimating an emotional state
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H20/00—ICT specially adapted for therapies or health-improving plans, e.g. for handling prescriptions, for steering therapy or for monitoring patient compliance
    • G16H20/70—ICT specially adapted for therapies or health-improving plans, e.g. for handling prescriptions, for steering therapy or for monitoring patient compliance relating to mental therapies, e.g. psychological therapy or autogenous training

Definitions

  • the method of providing emotional content consists of dividing the content in the text form of the diary by word and calculating the emotional level in each paragraph.
  • the contents of text contents such as online newspaper articles, e-mails, or diaries are stored in text form, and the stored contents are divided by word, and each word related to the emotional level (evaluation) is extracted from each paragraph.
  • a configuration is disclosed that finds word data stored for words that match each word, reads emotion values for each word, calculates emotion values for each paragraph, and then provides various types of avatars to subscribers.
  • Patent Document 1 Republic of Korea Patent Publication No. 10-2015-0145844
  • Patent Document 2 Republic of Korea Patent Publication No. 10-2019-0044809
  • the present invention is derived from such a technical background, and provides a service technology environment in which psychological tests can be performed by inducing convenient video and audio expressions of thoughts felt in daily life even for young children who lack reading comprehension and writing skills This is a video that can stabilize the psychology of children who are showing social anxiety through psychological testing and early diagnosis from infancy because reliable multi-faceted psychological analysis is possible by combining facial expressions, voices, and words just by registering a video diary.
  • An object of the present invention is to provide an apparatus and method for analyzing a psychological state using a diary.
  • Another object of the present invention is to provide an apparatus and method for analyzing a psychological state using an image diary for providing a service that allows a child to perform multi-faceted psychological analysis with minimal emotional expression at home without visiting an offline psychological counseling center.
  • young users write a video diary by recording a video diary as if talking to a video diary character friend of their thoughts during the day, and answering (psychological counseling) directly to the contents of the diary through the character selected.
  • the purpose is to provide an apparatus and method for analyzing a psychological state using a video diary that can be interesting and can induce richer emotional exposure through communication with characters.
  • the present invention for achieving the above object includes the following configuration.
  • the apparatus for analyzing a mental state using an image diary includes a data input unit that receives image diary data written by a video diary creator through a diary writing mode of an image diary dedicated application, and an image input to the data input unit It includes an analysis result providing unit that provides the psychological state analysis result analyzed by the psychological state analysis unit through the parental management mode of the video diary-only application and a psychological state analysis unit that analyzes the diary data to analyze the psychological state of the video diary creator .
  • the psychological state analysis method performed in the psychological state analysis apparatus including a data input unit, a psychological state analysis unit, and an analysis result providing unit
  • the data input unit is a video diary created by a video diary creator through a diary writing mode of a dedicated video diary application Step of receiving data, the psychological state analysis unit analyzing the image diary data input to the data input unit to analyze the psychological state of the image diary creator, and the analysis result providing unit the psychological state through the guardian management mode of the image diary dedicated application and providing a psychological state analysis result analyzed by the analysis unit.
  • the present invention by providing a service technology environment in which psychological tests can be conducted by inducing convenient video and audio expressions of thoughts felt in daily life even for young children who lack reading comprehension and writing skills, only registering a diary with video
  • a psychological state analysis device using an image diary that can stabilize the psychology of children who are showing social anxiety through psychological examination and early diagnosis from infancy and early diagnosis because reliable multi-faceted psychological analysis is possible by combining facial expressions, voices, and words. The effect that a method can be provided is derived.
  • the present invention can provide an apparatus and method for analyzing a psychological state using an image diary for providing a service that allows a child to perform multi-faceted psychological analysis with minimal emotional expression at home without visiting an offline psychological counseling center.
  • young users write a video diary by recording a video diary as if talking to a video diary character friend of their thoughts during the day, and answering (psychological counseling) directly to the contents of the diary through the character selected. It is possible to provide an apparatus and method for analyzing a psychological state using a video diary that can be interesting and can induce richer emotional exposure through communication with characters.
  • FIG. 1 is a block diagram showing the configuration of an apparatus for analyzing a psychological state using an image diary according to an embodiment of the present invention
  • FIG. 2 is an exemplary diagram for explaining a psychological analysis algorithm in a psychological state analysis unit according to an embodiment of the present invention
  • FIG. 3 is an exemplary diagram for explaining a process of deriving a product combining facial expression, voice, and word analysis results in a psychological state analysis unit according to an embodiment of the present invention
  • 4A to 4E are exemplary views of a screen provided in a parental management mode of a video diary-only application according to an embodiment of the present invention.
  • 5A and 5B are exemplary views of a screen provided in a diary writing mode of an image diary-only application according to an embodiment of the present invention
  • 6A to 6D are exemplary views of a screen provided in the counselor mode of the video diary-only application
  • FIG. 7 is a flowchart illustrating a method for analyzing a mental state using an image diary according to an embodiment of the present invention.
  • FIG. 1 is a block diagram illustrating the configuration of an apparatus for analyzing a psychological state using an image diary according to an embodiment of the present invention.
  • the psychological state analysis apparatus 10 communicates with the user terminal 20 and the psychological counselor terminal 30 .
  • the user terminal 20 and the psychological counselor terminal 30 are a smart phone, a portable terminal, a mobile terminal, a foldable terminal, a personal digital assistant: PDA), PMP (Portable Multimedia Player) terminal, Telematics terminal, navigation terminal, personal computer, notebook computer, slate PC, tablet PC, ultrabook ( ultrabook), wearable devices (including, for example, watch-type terminals (Smartwatch), glass-type terminals (Smart Glass), HMD (Head Mounted Display), etc.), Wibro terminals, IPTV (Internet Protocol Television) It can be applied to various terminals such as a terminal, a smart TV, a terminal for digital broadcasting, an audio video navigation (AVN) terminal, an audio/video (A/V) system, a flexible terminal, and a digital signage device.
  • the present invention is not limited thereto.
  • the user terminal 20 and the psychological counselor terminal 30 are driven with a dedicated app for psychological state analysis using an image diary according to an embodiment.
  • the user terminal 20 and the psychological counselor terminal 30 according to an embodiment have the same technical configuration, but the user terminal 20 when the video diary creator and their guardian possess the same, and the psychological counselor terminal in the case of the psychological counselor's possession (30) is divided and explained.
  • the psychological state analysis device 10 using an image diary is a questionnaire (survey) method as part of the 'image data-based AI service' to exclude 'Hangul reading ability and writing expression from student psychological counseling' It provides a 'video diary app' service for those aged 4 to 9 with inexperienced abilities.
  • psychological analysis is performed with artificial intelligence on the psychological state, emotional state, interest, and speed of speech, focusing on words extracted from facial expressions and voice only by video recording.
  • the apparatus 10 for analyzing a psychological state may perform psychological analysis by changing the words to analyzeable words through artificial intelligence learning even in the case of poor pronunciation of young children.
  • it can operate in conjunction with Google Cloud API, and when extracting an expression image from video data of a video diary, video, face recognition, and expression change extraction can be performed.
  • the psychological state analysis apparatus 10 may be utilized to develop an artificial intelligence chatbot service by using words and facial expression data used by age groups or collected from individual children. That is, by using the words that the video diary creator (child) who writes a video diary uses for psychological counseling conversation, it is possible to have a counseling conversation with a child according to the level of thinking ability, and furthermore, it is possible to apply it to a personality education service.
  • the apparatus 10 for analyzing a psychological state includes a communication unit 110 , a data input unit 120 , a psychological state analysis unit 130 , an analysis result providing unit 140 , and an information providing unit 150 . ), a character selection unit 160 , a feedback providing unit 170 , and a storage unit 180 .
  • the communication unit 110 communicates with any internal component or at least one external terminal through a wired/wireless communication network.
  • the external arbitrary terminal may include the user terminal 20 and the psychological counselor terminal 30 .
  • wireless Internet technologies wireless LAN (WLAN), DLNA (Digital Living Network Alliance), WiBro (Wireless Broadband: Wibro), WiMAX (World Interoperability for Microwave Access: Wimax), HSDPA (High Speed Downlink Packet Access) ), High Speed Uplink Packet Access (HSUPA), IEEE 802.16, Long Term Evolution (LTE), Long Term Evolution-Advanced (LTE-A), Wireless Mobile Broadband Service (WMBS), etc.
  • the communication unit 110 transmits and receives data according to at least one wireless Internet technology within a range including Internet technologies not listed above.
  • short-range communication technologies include Bluetooth, Radio Frequency Identification (RFID), Infrared Data Association (IrDA), Ultra Wideband (UWB), ZigBee, and Near Field Communication (NFC).
  • RFID Radio Frequency Identification
  • IrDA Infrared Data Association
  • UWB Ultra Wideband
  • ZigBee ZigBee
  • NFC Near Field Communication
  • USB Ultra Sound Communication
  • VLC Visible Light Communication
  • Wi-Fi Wi-Fi
  • Wi-Fi Direct Wi-Fi Direct
  • the wired communication technology may include Power Line Communication (PLC), USB communication, Ethernet, serial communication, optical/coaxial cable, and the like.
  • the communication unit 110 communicates with the user terminal 20 and the psychological counselor terminal 30 .
  • the data input unit 120 receives the video diary data written by the video diary creator through the diary writing mode of the video diary dedicated application.
  • the image diary data may be image data captured by the camera of the user terminal 20 .
  • the mental state analyzer 130 analyzes the image diary data input to the data input unit 120 to analyze the psychological state of the image diary creator.
  • the psychological state analysis unit 130 is a facial expression (PNG), voice (Mp3) and text morphemes around the reproduction section of the video diary data in the video (Mp4) format spoken by the video diary creator (child)
  • the result is calculated by applying the ratio (expression, word, voice order) to each tone emotion index and emotion attribute value for each word as priority.
  • the psychological state can be analyzed using the calculated result value. In other words, it categorizes the images in the video diary into facial expressions and voices, extracts pitches and words derived from voices, and produces psychological analysis results through artificial intelligence learning.
  • the mental state analyzer 130 includes an image analyzer 132 that analyzes image data included in the image diary data input to the data input unit 120 and the image diary data input to the data input unit 120 . and a voice analyzer 134 that analyzes voice data included in the .
  • the image analysis unit 132 recognizes the expression of the image diary creator from the image included in the image diary data.
  • the image analysis unit 132 can divide the psychological state of the video diary creator into joy, surprise, sadness, anger, interest, boredom, fear, pain, and neutrality, and the emotional state is excited, positive, negative, It can be classified as one of the stable states.
  • the image analysis unit 132 receives the object information of the object responding to the presented stimulus and uses the face recognition means for recognizing the face of the object, and the recognized face of the object. Facial element extraction means for extracting elements, and emotion recognition means for recognizing the emotion expressed by the object by recognizing a change in the facial expression of the object that changes in response to the stimulus based on the extracted facial element of the object.
  • the face recognition means uses the Adaboost learning algorithm to recognize the face of the object from the object information.
  • the Adaboost learning algorithm classifies the feature points of the extracted object using the Har-Like feature points.
  • the face element extraction means extracts the face element of the object by using the face of the object recognized by the face recognition means.
  • the extracted facial elements are both eyes, both eyebrows, the forehead, or the mouth.
  • the image analysis unit 132 recognizes a face region from the image diary data including the image diary creator input to the data input unit 120 , and extracts a facial element from the recognized face.
  • the Adaboost learning algorithm may be used to extract the face element of the object.
  • the emotion recognition means recognizes the emotion expressed by the object by recognizing a change in facial expression based on the facial elements (both eyes, both eyebrows, mouth) of the object extracted from the facial element extraction means.
  • the change in the subject's facial expression is the change in the size of the eyes, the change in the distance between the eyes and the eyebrows, the change in the shape of the forehead, the change in the size of the mouth, and the change in the shape of the mouth. , sadness, anger, interest, boredom, fear, pain, neutrality).
  • the present invention is not limited thereto.
  • the voice analyzer 134 includes a pitch analyzer 1342 that analyzes the pitch of the voice data, and a word analyzer 1344 that analyzes morphemes of the voice data to determine the frequency of words and analyzes the number of characters per hour. do.
  • the pitch analyzer 1342 may analyze pitch pitch and decibel to determine psychological states such as joy, fear, surprise, sadness, dislike, and anger, and emotional states such as excitement, positive, negative, and stable.
  • FIG. 2 is an exemplary diagram for explaining a psychological analysis algorithm in the psychological state analyzer according to an embodiment of the present invention.
  • the psychological state analysis unit 130 extracts the expression image (PNG), the voice (MP3), the word (TEXT), the pitch, and the playback time for each playback section in which a word is spoken in the child's video (Mp4). It is possible to extract text morphemes from the voice (Mp3) again (STT-Speech to Text), and reclassify them to the frequency of each word and the number of characters.
  • the psychological state and emotional state of the video diary creator are identified based on the facial expression image of the video diary data and the pitch of the voice.
  • voice analysis of the video diary data it is possible to determine the interest or speaking speed of the video diary creator based on word morphemes, frequency, and number of letters.
  • FIG. 3 is an exemplary diagram for explaining a process of deriving a product obtained by combining the facial expression, voice, and word analysis results in the psychological state analyzer according to an embodiment of the present invention.
  • a positive smiley expression and pitch for the word “good” in a way that numerically calculates the psychological change of the video diary contents, Set so that psychological analysis values such as speech speed exist.
  • psychological analysis values such as speech speed exist.
  • the same word “okay” is used in a specific period and negative analysis values are obtained that are different from usual in facial expression, pitch, and speed of speech, it can be implemented to diagnose anxiety symptoms.
  • the analysis result providing unit 140 is a psychological state of the video diary creator, the emotional state of the video diary creator, and the video diary creator according to the psychological state analysis result analyzed by the psychological state analysis unit 130 . It is characterized in that at least one of interest and the speed of speech of the video diary creator is provided as an analysis result.
  • the analysis result providing unit 140 provides the psychological state analysis result analyzed by the psychological state analysis unit 130 through the guardian management mode of the video diary-only application.
  • the analysis result providing unit 140 may provide the analysis result by classifying the psychological state into negative, positive, stable, etc. as shown in FIG. 3 .
  • 4A to 4E are exemplary views of a screen provided in a parental management mode of a video diary-only application according to an embodiment of the present invention.
  • the parental management mode of the video diary-only application may list the video diary data of the children of the guardians as shown in FIG. 4A and provide the screen in the form of a thumbnail.
  • the child's name may be included, and if the summary screen is clicked on the screen listed as shown in FIG. 4A, the video diary data may be reproduced.
  • the voice of the video diary data may be recognized and provided in a text format.
  • the analysis result providing unit 140 may classify the emotional state into two categories according to the analysis result of the video diary data as shown in FIG. 4C and provide the divided emotion and positive/negative items.
  • the analysis results are graphed so that you can understand which emotion is the most important among joy, fear, surprise, sadness, dislike, and anger, and check whether there are many positive or negative factors, excited or stable at a glance. It can be provided by conciliation.
  • the video diary creator's interest as shown in FIG. 4D.
  • the degree of interest may be differentially displayed and provided, or may be provided by sorting in order of word exposure. Accordingly, it can help the parenting by accurately judging the interests of the children.
  • the speed of speech may be calculated to one decimal place as the number of characters/playback time (s).
  • the video diary-only application may arbitrarily adjust a playback section, a playback time, and a playback speed when the video diary data is played back.
  • the information providing unit 150 receives and provides a comment from the psychological counselor terminal 30 on the analysis result provided by the analysis result providing unit 140 .
  • the analysis result providing unit 140 is a technical configuration that provides a psychological state analysis result by artificial intelligence, while the information providing unit 150 is based on the image diary data input to the data input unit 120 at least one or more psychological It is a technical configuration that provides psychological analysis comments of a plurality of psychological counselors from the counselor terminal 30 . In this regard, it is possible to accurately grasp the psychological state by collecting the opinions of various psychological counselors, even without face-to-face counseling.
  • the character selector 160 provides at least one character through the diary writing mode of the video diary-only application, and receives one of the provided characters selected.
  • the feedback providing unit 170 is the psychological state of the video diary creator in a conversational manner with the character selected in the character selector 160 in the diary writing mode of the video diary-only application. Provides feedback on status. That is, by using the selected character image and voice, it is possible to provide feedback in a conversational manner with animation effects applied on behalf of the psychological counselor.
  • the video diary creator child
  • 5A and 5B are exemplary views of a screen provided in a diary writing mode of an image diary-only application according to an embodiment of the present invention.
  • the character selector 160 may provide at least one character in the diary writing mode as shown in FIG. 5A , and may receive one of the characters selected from the video diary creator.
  • the video diary creator may be induced to additionally write a video diary in a dialogue format with the selected character, or psychological counseling may be performed according to the video diary analysis result. Accordingly, it is possible to grasp the detailed psychological state and to take action according to the psychological state.
  • the feedback providing unit 170 receives a feedback according to the psychological state analysis result from the psychological counselor according to the psychological state analysis result provided by the analysis result providing unit in the psychological counselor mode of the video diary-only application.
  • the feedback providing unit 170 may output the psychological state analysis result from the psychological counselor or additional questions for in-depth analysis in the form of conversation with the character as shown in FIG. 5B .
  • 6A to 6D are exemplary views of a screen provided in the counselor mode of the video diary-only application.
  • the psychological counselor can check the video diary data and input a comment in the counselor mode of the video diary dedicated application.
  • the psychological counselor may provide and manage feedback through psychological analysis for a plurality of video diary writers (students) as shown in FIG. 6A .
  • the video diary-only application displays the students managed in the counselor mode by age, or displays the list of students by various criteria such as recent subscription order, diary registration date order, age group (months) order, name order It can be provided by sorting.
  • the video diary-only application provides a list of video diaries by date for each student being managed as shown in FIG. 6b in the counselor mode, and the psychological counselor checks the video diary that needs feedback and then inputs the feedback. Provides a user interface.
  • the psychological counselor in charge may check the results of artificial intelligence analysis on the emotional state of the video diary that is managed and write an evaluation opinion in addition thereto. That is, according to one embodiment, it is possible to provide the final analysis result to the guardian of the video diary creator by reflecting the evaluation result of the psychological counselor in the artificial intelligence psychological analysis result for the video diary. Accordingly, the accuracy of psychological analysis can be improved, and the effect of being able to judge by collecting opinions of many experts is derived even if the guardian does not personally visit the counseling center offline.
  • the storage unit 180 is a flash memory type (Flash Memory Type), a hard disk type (Hard Disk Type), a multimedia card micro type (Multimedia Card Micro Type), a card type memory (eg, SD or XD memory, etc.) ), magnetic memory, magnetic disk, optical disk, RAM (Random Access Memory: RAM), SRAM (Static Random Access Memory), ROM (Read-Only Memory: ROM), EEPROM (Electrically Erasable Programmable Read-Only Memory), PROM ( Programmable Read-Only Memory) may include at least one storage medium.
  • the storage unit 180 stores data necessary for running and updating an app dedicated to mental state analysis using an image diary.
  • it stores user information and psychological counselor information registered as a member through a dedicated app for psychological state analysis using a video diary.
  • the present invention is not limited thereto.
  • FIG. 7 is a flowchart illustrating a method for analyzing a mental state using an image diary according to an embodiment of the present invention.
  • the psychological state analysis method performed in the psychological state analysis apparatus including the data input unit, the psychological state analyzer, and the analysis result providing unit is first, the data input unit to the video diary creator through the diary writing mode of the video diary dedicated application.
  • the video diary data created by the user is input (S710).
  • the image diary data may be image data captured by the camera of the user terminal 20 .
  • the character selector before receiving the image diary data, the character selector provides at least one character through the diary writing mode of the image diary dedicated application, and receives one of the provided characters selected (S700).
  • the psychological state analyzer analyzes the image diary data input to the data input unit to analyze the psychological state of the image diary creator (S720).
  • analyzing the psychological state extracts facial expressions (PNG), voice (Mp3) and text morphemes based on the reproduction section of the video diary data in the video (Mp4) format spoken by the video diary creator (child)
  • the result value is calculated by applying the ratio (expression, word, and voice order) to each pitch emotion index and emotion attribute value for each word as priority.
  • the psychological state can be analyzed using the calculated result value. In other words, it categorizes the images in the video diary into facial expressions and voices, extracts pitches and words derived from voices, and produces psychological analysis results through artificial intelligence learning.
  • the psychological state analysis unit can classify the psychological state of the video diary creator into joy, surprise, sadness, anger, interest, boredom, fear, pain, and neutrality, and can classify the emotional state into one of excited, positive, negative, and stable states.
  • the feedback providing unit provides feedback on the psychological state of the video diary creator in a conversational manner with the character selected in the character selection unit in the diary writing mode of the video diary dedicated application according to the psychological state analysis result analyzed by the psychological state analysis unit. (S730).
  • the video diary creator (child) can present a diary topic that can be healed by itself, or send praise stickers such as comfort, praise, hope, etc. depending on the diary contents.
  • the analysis result providing unit provides the psychological state analysis result analyzed by the psychological state analyzer through the guardian management mode of the video diary-only application (S740).
  • the psychological state analysis result provides at least one of the psychological state of the video diary creator, the emotional state of the video diary creator, the interest of the video diary creator, and the speed of speech of the video diary creator as the analysis result according to the psychological state analysis result.
  • the above-described method may be implemented as an application or implemented in the form of program instructions that may be executed through various computer components and recorded in a computer-readable recording medium.
  • the computer-readable recording medium may include program instructions, data files, data structures, etc. alone or in combination.
  • the program instructions recorded in the computer-readable recording medium are specially designed and configured for the present invention, and may be known and available to those skilled in the computer software field.
  • Examples of the computer-readable recording medium include hard disks, magnetic media such as floppy disks and magnetic tapes, optical recording media such as CD-ROMs and DVDs, and magneto-optical media such as floppy disks. media), and hardware devices specially configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like.
  • Examples of program instructions include not only machine language codes such as those generated by a compiler, but also high-level language codes that can be executed by a computer using an interpreter or the like.
  • the hardware device may be configured to operate as one or more software modules for carrying out the processing according to the present invention, and vice versa.

Landscapes

  • Health & Medical Sciences (AREA)
  • Engineering & Computer Science (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Physics & Mathematics (AREA)
  • General Health & Medical Sciences (AREA)
  • Public Health (AREA)
  • Medical Informatics (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Veterinary Medicine (AREA)
  • Animal Behavior & Ethology (AREA)
  • Surgery (AREA)
  • Psychiatry (AREA)
  • Molecular Biology (AREA)
  • Biomedical Technology (AREA)
  • Pathology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Hospice & Palliative Care (AREA)
  • Human Computer Interaction (AREA)
  • Multimedia (AREA)
  • Acoustics & Sound (AREA)
  • Child & Adolescent Psychology (AREA)
  • Developmental Disabilities (AREA)
  • Psychology (AREA)
  • Social Psychology (AREA)
  • Signal Processing (AREA)
  • Artificial Intelligence (AREA)
  • Primary Health Care (AREA)
  • Epidemiology (AREA)
  • Educational Technology (AREA)
  • Nursing (AREA)
  • Measurement Of The Respiration, Hearing Ability, Form, And Blood Characteristics Of Living Organisms (AREA)

Abstract

La présente invention concerne un appareil et un procédé d'analyse d'état mental utilisant un agenda vidéo, qui utilise une application dédiée à l'analyse d'état mental à l'aide d'un journal vidéo exécuté par un terminal utilisateur, selon la présente invention, un environnement technologique de service capable de réaliser un test psychologique en amenant facilement enfant, même jeune, qui a une capacité de compréhension de lecture et une écriture insuffisantes à exprimer des pensées concernant une vie normale dans une vidéo et une voix pouvant être fournie par l'appareil d'analyse d'état mental utilisant un agenda vidéo, comprenant : une unité d'entrée de données pour recevoir une entrée de données d'agenda vidéo préparées par un préparateur d'agenda vidéo par l'intermédiaire d'un mode de préparation d'agenda d'une application dédiée à un agenda vidéo ; une unité d'analyse d'état mental pour analyser un état mental du préparateur d'agenda vidéo par analyse des données d'agenda vidéo entrées dans l'unité d'entrée de données ; et une unité de fourniture de résultat d'analyse pour fournir un résultat d'analyse de l'état mental analysé par l'unité d'analyse d'état mental par l'intermédiaire d'un mode de gestion sécurisé de l'application dédiée à un agenda vidéo.
PCT/KR2021/007826 2020-06-22 2021-06-22 Appareil et procédé d'analyse d'état mental utilisant un agenda vidéo WO2021261887A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
KR1020200075990A KR102187210B1 (ko) 2020-06-22 2020-06-22 영상 일기를 이용한 심리 상태 분석 장치 및 방법
KR10-2020-0075990 2020-06-22

Publications (1)

Publication Number Publication Date
WO2021261887A1 true WO2021261887A1 (fr) 2021-12-30

Family

ID=73776847

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/KR2021/007826 WO2021261887A1 (fr) 2020-06-22 2021-06-22 Appareil et procédé d'analyse d'état mental utilisant un agenda vidéo

Country Status (2)

Country Link
KR (1) KR102187210B1 (fr)
WO (1) WO2021261887A1 (fr)

Families Citing this family (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR102187210B1 (ko) * 2020-06-22 2020-12-04 주식회사 플랙스 영상 일기를 이용한 심리 상태 분석 장치 및 방법
KR102670492B1 (ko) * 2021-01-21 2024-05-30 주식회사 에스알유니버스 인공지능을 활용한 심리 상담 방법 및 장치
KR102476787B1 (ko) * 2022-02-14 2022-12-12 주식회사 플랙스 인공 지능 모델 기반으로 그림일기 분석을 통한 그림일기 사용자의 심리상태 진단 장치 및 방법
KR20240014154A (ko) * 2022-07-25 2024-02-01 테바소프트 주식회사 음성인식을 활용한 온라인 심리상담 시스템
KR102514357B1 (ko) 2022-12-06 2023-03-27 김미양 스티커 부착을 통한 피검사자의 심리 상태 판단 방법, 장치 및 시스템
KR102514361B1 (ko) 2022-12-26 2023-03-27 김미양 스티커 부착을 통한 피검사자의 심리 검사를 제공하는 교재를 결정하고 이를 이용하여 심리 검사를 수행하는 방법, 장치 및 시스템
KR102642997B1 (ko) * 2023-10-12 2024-03-04 지상선 정신 건강을 위한 자아 활성화 시스템

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20140032651A (ko) * 2012-09-07 2014-03-17 전자부품연구원 감성 피드백 서비스 방법 및 이를 적용한 스마트 디바이스
KR101581801B1 (ko) * 2015-08-07 2016-01-04 주식회사 티아이런 캐릭터를 이용한 3자통신 기반 아동심리 상담치료 시스템 및 방법
KR20170013015A (ko) * 2015-07-27 2017-02-06 주식회사 프론트유 스마트 완구 기반 감성 커뮤니케이션 시스템
KR20190118139A (ko) * 2018-04-09 2019-10-17 주식회사 마이다스아이티 영상 분석 시스템 및 그 방법
KR20200023092A (ko) * 2018-08-24 2020-03-04 십일번가 주식회사 심리 분석 기반의 이상 상태 감지 방법 및 장치
KR102187210B1 (ko) * 2020-06-22 2020-12-04 주식회사 플랙스 영상 일기를 이용한 심리 상태 분석 장치 및 방법

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR101661698B1 (ko) 2014-06-19 2016-10-05 오미라 도형 및 문항 검사를 통한 온라인 심리분석 서비스 방법
KR102005437B1 (ko) 2017-10-23 2019-10-01 (주)에스오피에스 증강현실 기반 아동용 컬러링북을 이용한 심리분석 시스템 및 그 시스템을 이용한 분석방법

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20140032651A (ko) * 2012-09-07 2014-03-17 전자부품연구원 감성 피드백 서비스 방법 및 이를 적용한 스마트 디바이스
KR20170013015A (ko) * 2015-07-27 2017-02-06 주식회사 프론트유 스마트 완구 기반 감성 커뮤니케이션 시스템
KR101581801B1 (ko) * 2015-08-07 2016-01-04 주식회사 티아이런 캐릭터를 이용한 3자통신 기반 아동심리 상담치료 시스템 및 방법
KR20190118139A (ko) * 2018-04-09 2019-10-17 주식회사 마이다스아이티 영상 분석 시스템 및 그 방법
KR20200023092A (ko) * 2018-08-24 2020-03-04 십일번가 주식회사 심리 분석 기반의 이상 상태 감지 방법 및 장치
KR102187210B1 (ko) * 2020-06-22 2020-12-04 주식회사 플랙스 영상 일기를 이용한 심리 상태 분석 장치 및 방법

Also Published As

Publication number Publication date
KR102187210B1 (ko) 2020-12-04

Similar Documents

Publication Publication Date Title
WO2021261887A1 (fr) Appareil et procédé d'analyse d'état mental utilisant un agenda vidéo
Walter‐Laager et al. Media‐assisted language learning for young children: Effects of a word‐learning app on the vocabulary acquisition of two‐year‐olds
Motha et al. “Non‐coercive rearrangements”: Theorizing desire in TESOL
Dicks et al. Multimodality and ethnography: working at the intersection
Henze et al. Informal teaching and learning: A study of everyday cognition in a Greek community
Martin " Always in my face": An exploration of social class consciousness, salience, and values
WO2013085320A1 (fr) Procédé permettant de fournir un service d'acquisition et d'étude d'une langue étrangère d'après une reconnaissance de contexte au moyen d'un dispositif intelligent
Souto-Manning Competence as linguistic alignment: Linguistic diversities, affinity groups, and the politics of educational success
Hyter Pragmatic assessment and intervention in children
Cartmill Mind the gap: Assessing and addressing the word gap in early education
Kim The role of peer relationships and interactions in preschool bilingual children's responses to picture books
Lico et al. An important part of me: A dialogue about difference
Khan Multimodal behavioral analytics in intelligent learning and assessment systems
Cheng et al. Investigating mobile emotional learning for children with autistic spectrum disorders
Bhatia Using transfer learning, spectrogram audio classification, and MIT app inventor to facilitate machine learning understanding
Peltokorpi et al. Effect of tactile imitation guidance on imitation and emotional availability. A case report of a mother and her child with congenital deafblindness
KR20210053661A (ko) 심리케어 시스템 및 그 방법
Li et al. Towards a novel machine learning approach to support augmentative and alternative communication (AAC)
WO2022203123A1 (fr) Procédé et dispositif de fourniture d'un contenu d'enseignement vidéo sur la base d'un traitement de langage naturel par l'intelligence artificielle au moyen d'un personnage
Topuz et al. Script fading procedure: A systematic review and meta-analysis
CN115222857A (zh) 生成虚拟形象的方法、装置、电子设备和计算机可读介质
WO2023079370A1 (fr) Système et procédé pour améliorer la qualité d'une expérience d'apprentissage/enseignement
Poovizhi et al. M-Learning First Word Android Application to support Autistic Children with their Education
Strong-Wilson et al. Locked in and locked out: Covid-19 and teaching “remotely”
Nash-Ditzel et al. Freedoms in the classroom: Cultivating a successful third space for literacy growth

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21828124

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205N DATED 19/05/2023)

122 Ep: pct application non-entry in european phase

Ref document number: 21828124

Country of ref document: EP

Kind code of ref document: A1