WO2021118084A2 - Breast cancer self-examination server and method therefor - Google Patents

Breast cancer self-examination server and method therefor Download PDF

Info

Publication number
WO2021118084A2
WO2021118084A2 PCT/KR2020/016073 KR2020016073W WO2021118084A2 WO 2021118084 A2 WO2021118084 A2 WO 2021118084A2 KR 2020016073 W KR2020016073 W KR 2020016073W WO 2021118084 A2 WO2021118084 A2 WO 2021118084A2
Authority
WO
WIPO (PCT)
Prior art keywords
breast
examination
self
breast cancer
user terminal
Prior art date
Application number
PCT/KR2020/016073
Other languages
French (fr)
Korean (ko)
Other versions
WO2021118084A3 (en
Inventor
김승미
Original Assignee
(주)규리아
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by (주)규리아 filed Critical (주)규리아
Publication of WO2021118084A2 publication Critical patent/WO2021118084A2/en
Publication of WO2021118084A3 publication Critical patent/WO2021118084A3/en

Links

Images

Classifications

    • GPHYSICS
    • G16INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16HHEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H50/00ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics
    • G16H50/20ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics for computer-aided diagnosis, e.g. based on medical expert systems
    • GPHYSICS
    • G16INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16HHEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H30/00ICT specially adapted for the handling or processing of medical images
    • G16H30/20ICT specially adapted for the handling or processing of medical images for handling medical images, e.g. DICOM, HL7 or PACS
    • GPHYSICS
    • G16INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16HHEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H50/00ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics
    • G16H50/30ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics for calculating health indices; for individual health risk assessment

Definitions

  • the breast cancer self-examination server and method according to the present invention relates to a breast cancer self-examination server and method for self-examing breast cancer using a user terminal without visiting a hospital and providing whether or not treatment is required.
  • breast cancer is one of the cancers with a high incidence rate, occupying the fifth place among all cancers. Almost all breast cancer patients are women, and in particular, among female cancers (except for thyroid cancer), the incidence rate is the highest. In terms of age group, the number of breast cancer patients was the highest in their 40s, followed by those in their 50s and 60s.
  • women over the age of 40 are screened for breast cancer using X-ray mammography.
  • a tissue suspected of being cancerous is found as a result of X-ray mammography
  • the patient is subjected to a precise diagnosis.
  • An object of the present invention is to provide a breast cancer self-examination server and a method therefor.
  • an object of the present invention is to assist a user in early detection of breast cancer by performing self-examination of breast cancer in a convenient way by a user using a mobile terminal and providing information on the need for medical treatment or precise diagnosis as a result of the self-examination.
  • a breast cancer self-examination method using a breast cancer self-examination server comprising: receiving health information from a user terminal; transmitting breast image capturing guide information using augmented reality to the user terminal; receiving a breast image photographed according to the transmitted guide information from the user terminal; transmitting a breast self-examination record unit to the user terminal that has transmitted the breast image; receiving a breast self-examination record recorded in the breast self-examination record unit from a user terminal that has received the breast self-examination record unit; and transmitting, to the user terminal, a checkup opinion including an opinion on the need for consultation with a medical staff provided based on the health information received from the user terminal, a breast image, and a breast cancer self-examination record, and an opinion on the need for a detailed examination to the user terminal.
  • a checkup opinion including an opinion on the need for consultation with a medical staff provided based on the health information received from the user terminal, a breast image, and a breast cancer self-examination record, and an
  • the health information may provide a breast cancer self-examination method including questionnaire information related to breast cancer.
  • the breast cancer-related questionnaire information includes breast cancer self-examination time, menstruation, breast self-examination time based on menstruation date, breast cancer family history, childbirth, first childbirth after 30 years of age, obesity, female hormone administration It is possible to provide a breast cancer self-examination method, including at least one of whether or not the breast cancer has been treated with breast radiation, and whether the breast cancer is malignant.
  • the breast imaging guide information may provide a breast cancer self-examination method, including voice information and image information indicating a breast palpation method.
  • the breast cancer self-examination method may be provided, wherein the voice information includes a breast palpation method guide voice output by matching a screen output according to the breast palpation order.
  • the image information may provide a breast cancer self-examination method including at least one of a breast placement guide display and a breast promotion direction guide display displayed on a camera photographing screen of the user terminal.
  • the user's breast displayed on the camera screen is recognized through a deep learning model learned based on the previously collected breast image data, and the breast image using the augmented reality based on the recognized breast image
  • the pre-collected breast image data is breast image data labeled with any one of the full range of the breast, nipples, areolas, clavicle, and armpit.
  • the examination opinion when the examination opinion is any one of the need for consultation with a medical team or the need for a detailed examination, the examination opinion for transmitting the examination opinion to the hospital guide information close to the screen of the user terminal, the hospital call connection unit, and the hospital server It is possible to provide a breast cancer self-examination method in which a user interface including a transmission unit is provided.
  • the self-examination record may be displayed on the breast image and include a mark having a different shape according to breast cancer symptom information, the breast cancer self-examination method may be provided.
  • the breast cancer symptom information includes at least one of a painless lump, a lump in the breast and armpit, a bloody discharge from the nipple, eczema of the nipple, a change in breast skin, a nipple depression, a change in breast size, a change in nipple position, and breast pain. It is possible to provide a breast cancer self-examination method, including any one.
  • a scheduled examination date is calculated based on the date the breast image is received from the user terminal and menstrual information included in the health information, and a notification is sent to the user terminal in response to the scheduled examination date It is possible to provide a breast cancer self-examination method.
  • a breast cancer self-examination server comprising: a memory in which a breast cancer self-examination program is recorded; and a processor executing the program recorded in the memory, wherein the processor transmits breast image photographing guide information using augmented reality to the user terminal according to the execution of the program, and photographing according to the transmitted guide information receiving a breast image from the user terminal, transmitting a breast self-examination record to the user terminal that has transmitted the breast image, and receiving the breast self-examination record recorded in the breast self-examination record from the user terminal receiving the breast self-examination record
  • a checkup opinion including an opinion on the need for medical staff consultation and an opinion on the need for a detailed examination provided based on the health information received from the user terminal, a breast image, and a breast cancer self-examination record We want to provide a checkup server.
  • a user can perform breast cancer self-examination in a convenient way using a mobile terminal, and as a result of the self-examination, information on the need for medical treatment or precise diagnosis can be provided to help early detection of breast cancer.
  • the present invention interlocks with the camera of the mobile terminal to allow the user to perform breast cancer self-examination by using the breast cancer self-examination sequence audio and the breast cancer screening sequence guide displayed on the recording screen and the breast promotion direction guide implemented in augmented reality. can be done easily.
  • the present invention analyzes a user's breast image using a deep learning model that has learned the breast image, and can provide the user with information on the need for more accurate breast cancer treatment.
  • FIG. 1 is a configuration diagram showing the configuration of a breast cancer self-examination server according to an embodiment of the present invention.
  • FIG. 2 is a flowchart of a breast cancer self-examination program according to an embodiment of the present invention.
  • FIG. 3 is a flowchart illustrating a breast cancer self-examination method according to an embodiment of the present invention.
  • FIG. 4 is a diagram illustrating a breast cancer self-examination program login screen and a main screen interface according to an embodiment of the present invention.
  • FIG. 5 is a diagram illustrating an example of a health information input item according to an embodiment of the present invention.
  • FIGS. 6 to 9 are diagrams illustrating a breast image capturing guide user interface according to an embodiment of the present invention.
  • FIG. 10 is a view showing a user self-examination record log according to an embodiment of the present invention.
  • FIG. 11 is a view showing a mark displayed on a breast image in a breast cancer self-examination recorder according to an embodiment of the present invention.
  • FIG. 1 is a configuration diagram showing the configuration of a breast cancer self-examination server 100 according to an embodiment of the present invention.
  • 2 is a flowchart of a breast cancer self-examination program according to an embodiment of the present invention.
  • the breast cancer self-examination server 100 utilizes a camera according to the execution of the breast cancer self-examination program of the user terminal to allow the user to voluntarily exist in the breast, such as lumps, tumors, etc.
  • a camera according to the execution of the breast cancer self-examination program of the user terminal to allow the user to voluntarily exist in the breast, such as lumps, tumors, etc.
  • self-examination of breast cancer accompanied by voice guidance through a speaker that matches the video guidance using deep learning and augmented reality (AR) technology is implemented. server to do it.
  • AR augmented reality
  • the breast cancer self-examination server 100 receives health information through communication between user terminals, photographing and recording breast cancer self-examination images, and whether treatment is necessary based on health information scores and breast cancer self-examination record scores The process of judgment and hospital linkage can be performed, and the detailed method will be described later.
  • the breast cancer self-examination server 100 may include a communication module 110 , a memory 120 , a processor 130 , and a database (DB: 140 ).
  • the communication module 110 provides a communication interface to the breast cancer self-examination server 100 by interworking with a communication network, and may serve to transmit and receive data to and from a user terminal.
  • the communication module 110 may be a device including hardware and software necessary for transmitting and receiving signals such as control signals or data signals through wired/wireless connection with other network devices.
  • the memory 120 may record a breast cancer self-examination method program. In addition, the memory 120 may perform a function of temporarily or permanently storing data processed by the processor 130 .
  • the memory 120 may include a volatile storage medium or a non-volatile storage medium, but the scope of the present invention is not limited thereto.
  • the processor 130 may control the entire process performed by the breast cancer self-examination method program in the breast cancer self-examination server 100 . Each step of the process performed by the processor 130 will be described later with reference to FIGS. 3 to 11 .
  • the processor 130 may include all kinds of devices capable of processing data, such as a processor.
  • the 'processor' may refer to a data processing device embedded in hardware, for example, having a physically structured circuit to perform a function expressed as a code or an instruction included in a program.
  • a microprocessor a central processing unit (CPU), a processor core, a multiprocessor, an application-specific integrated (ASIC) circuit
  • ASIC application-specific integrated
  • FPGA field programmable gate array
  • the database 140 may store data necessary for executing a breast cancer self-examination program, including breast image data learned by machine learning to be described later.
  • a 'terminal' may be a wireless communication device with guaranteed portability and mobility, for example, any type of handheld-based wireless communication device such as a smart phone, tablet PC, or notebook computer. have.
  • the 'terminal' may be a wired communication device such as a PC that can be connected to another terminal or a server through a network.
  • the network refers to a connection structure capable of exchanging information between each node, such as terminals and servers, and includes a local area network (LAN), a wide area network (WAN), and the Internet (WWW). : World Wide Web), wired and wireless data networks, telephone networks, and wired and wireless television networks.
  • LAN local area network
  • WAN wide area network
  • WWW Internet
  • wireless data communication networks examples include 3G, 4G, 5G, 3rd Generation Partnership Project (3GPP), Long Term Evolution (LTE), World Interoperability for Microwave Access (WIMAX), Wi-Fi, Bluetooth communication, infrared communication, ultrasound Communication, Visible Light Communication (VLC), LiFi, etc. are included, but are not limited thereto.
  • 3GPP 3rd Generation Partnership Project
  • LTE Long Term Evolution
  • WIMAX World Interoperability for Microwave Access
  • Wi-Fi Bluetooth communication
  • infrared communication ultrasound Communication
  • VLC Visible Light Communication
  • LiFi etc.
  • FIG. 3 is a flowchart illustrating a breast cancer self-examination method according to an embodiment of the present invention.
  • 4 is a diagram illustrating a breast cancer self-examination program login screen and a main screen interface according to an embodiment of the present invention.
  • 5 is a diagram illustrating an example of a health information input item according to an embodiment of the present invention.
  • receiving health information from a user terminal S210 may be performed.
  • the user may log in through the user interface of the breast cancer self-examination program on the terminal (see FIG. 4A ).
  • the user interface may display the number of days elapsed from the date of the last breast cancer self-examination to the present, and output an examination start button, expert consultation button, record view button, and a quick know button providing breast information (FIG. 4) of (b)).
  • Breast information may include the definition of breast cancer, statistics, causes and symptoms, the need for breast cancer self-examination, and methods for preventing breast cancer.
  • the user may first input health information into the breast cancer self-examination program of the terminal.
  • the health information may include breast cancer-related questionnaire information.
  • Breast cancer-related questionnaire information includes breast cancer self-examination time, menstruation status, breast self-examination time based on menstruation date, family history of breast cancer, childbirth, first childbirth after 30 years of age, obesity, female hormone administration, breast radiation treatment, and breast It may include at least one of malignant tumors.
  • FIGS. 6 to 9 are diagrams illustrating a breast image capturing guide user interface according to an embodiment of the present invention.
  • a step ( S220 ) of transmitting breast image capturing guide information using augmented reality to the user terminal may be performed.
  • the breast imaging guide information may include voice information and image information indicating a breast palpation method.
  • the image information may include at least one of a breast placement guide display and a breast promotion direction guide display displayed on a camera photographing screen of the user terminal.
  • the user may click the measure button on the interface of the terminal to start the breast self-examination, or click the usage guide button to receive a usage guide (refer to FIG. 6(a)).
  • the measurement button is clicked, the user is guided to place the chest of the user on the line displayed in augmented reality on the camera shooting screen of the terminal, and a measurement start button may be provided (see (b) of FIG. 6).
  • a finish button for storing the mammogram image and a journal entry button for writing a checkup record may be output (refer to FIG. 6(c) ).
  • the voice information informing the breast palpation method may be output so that the voice and the reference guidance screen are matched in the following order.
  • a guide voice indicating the start of promotion may be output.
  • a guide voice for horizontally fixing the user terminal to a desk, a dressing table, a bed, etc. for taking a photo and/or an image may be output.
  • the guidance screen shown in FIG. 6A may be output.
  • a camera focal length adjustment guide voice may be output to appear from both shoulders of the user to the navel on the camera photographing screen of the user terminal.
  • the guidance screen shown in FIG. 6B may be output.
  • a guide voice instructing the user to observe a change in the shape or contour of the user's breast displayed on the camera screen may be output from the user terminal.
  • a method of observing the breast with both arms down see Fig. 6 (c)
  • a method of observing the breast with both hands interlaced behind the head see Fig. 6 (d)
  • a voice guidance and a guidance screen for a method of observing the breast with both hands on the waist may be temporarily output.
  • the following procedure when the user performs the breast self-examination in a standing or sitting state, the following procedure may be performed.
  • a guidance voice output to align one breast with the circle displayed on the screen in a standing or sitting state, and a guidance screen to place the user's breast in the breast recognition display implemented in augmented reality on the screen may be output (FIG. 8) see (a)).
  • the user terminal may recognize it and switch to the next screen.
  • a voice and a screen for guiding to raise the arm on the side of the breast to be examined over the head and palpate it using the bottom surface of the first joint of the 2nd, 3rd, and 4th fingers of the opposite hand may be output (refer to (b) of FIG. 8).
  • a breast promotion direction guide display implemented in augmented reality may be output on the camera shooting screen to draw a circle from the outer upper part around the breast and examine the breast from the top, bottom, part, and under the armpit inward (FIG. 8). see (c)).
  • the following procedure may be performed.
  • a voice guiding the user to align one breast with the circle displayed on the screen in a lying state and a screen guiding the user to place the user's breast in the breast recognition display implemented in augmented reality on the screen may be output (FIG. 9 (a) ) Reference).
  • FIG. 9 (a) A voice guiding the user to align one breast with the circle displayed on the screen in a lying state and a screen guiding the user to place the user's breast in the breast recognition display implemented in augmented reality on the screen may be output (FIG. 9 (a) ) Reference).
  • the user places the breast in the breast recognition display it may be switched to the next screen.
  • a voice and a screen guiding the user to lie down and raise the arm on the side with the breast to be examined and to facilitate it with the opposite hand may be output (refer to FIG. 9(b) ).
  • a voice and a screen for guiding to raise the arm on the side of the breast to be examined over the head and palpate it using the bottom surface of the first joint of the 2nd, 3rd, and 4th fingers of the opposite hand may be output (refer to FIG. 9(c)).
  • a breast promotion direction guide display implemented in augmented reality may be output on the screen to draw a circle from the outer upper part around the breast and check from the upper, lower, and armpit areas inward (see FIG. 9 ( see d)).
  • a guide voice requesting to store the promotion image and fill out a questionnaire may be output.
  • breast cancer self-examination guide audio, guidance screen, and augmented reality display of breast placement guide and breast promotion direction guide display the user can follow the mark on the breast to promote it, and correct information about breast cancer and self-examination methods. You can easily learn information and do self-examination in a way that suits your breast.
  • the breast image capturing guide information using augmented reality may recognize the user's breast displayed on the camera screen through a deep learning model learned based on pre-collected breast image data. Adjacent to the recognized breast in this way, a breast image capturing and facilitation guide display implemented in augmented reality may be provided.
  • the breast placement guide display and the breast promotion direction guide display implemented by the above-described augmented reality may be included in the breast image capturing guide information.
  • the pre-collected breast image data may be breast image data labeled with any one of the full range of the breast, nipple, areola, clavicle, and armpit.
  • the deep learning model can automatically segment and extract the breast and other breast cancer-related areas on the breast image, and remove false detections outside the other areas.
  • Deep learning models can learn breast images using deep learning models such as Deconvolution Network and U-Net specialized in securing, segmenting and extracting Region of Interest (RoI) for breast and other regions. have.
  • deep learning models can be validated using data that did not participate in training to check their validity.
  • image extraction accuracy can be verified through statistical technique verification. K-fold cross-validation can be performed to verify objective generalization performance including overfitting of the trained deep learning model.
  • the detection model can verify and evaluate the performance of the model through Sensitivity and FPs/image.
  • data preprocessing and deep learning model learning/testing/verification are performed using a deep learning model, and the user's breast image is accurately recognized through the deep learning model.
  • Augmented reality breast image shooting guide information may be output on the screen.
  • a step (S230) of receiving the captured breast image from the user terminal according to the transmitted guide information may be performed.
  • FIG. 10 is a view showing a user self-examination record log according to an embodiment of the present invention.
  • a step S240 of transmitting the breast self-examination record to the user terminal that has transmitted the breast image may be performed.
  • the user can create a logbook after the breast cancer self-examination and set a reminder for the breast cancer self-exam schedule to build his/her own breast data and make self-examination a habit.
  • the breast cancer self-examination server 100 may calculate the scheduled examination date based on the date of receiving the breast image from the user terminal and menstruation information included in the health information, and transmit a notification to the user terminal in response to the scheduled examination date. have.
  • a date may be input on the user interface of the user terminal (refer to (a) of FIG. 9), and a screen (refer to (b) and (c) of FIG. 9) for writing a checkup diary may be output.
  • the user may record the examination log here.
  • FIG. 11 is a view showing a mark displayed on a breast image in a breast cancer self-examination recorder according to an embodiment of the present invention.
  • receiving the breast self-examination record recorded in the breast self-examination recorder from the user terminal receiving the breast self-examination record ( S250 ) may be performed.
  • the self-examination record may be displayed on the breast image and include marks having different shapes according to breast cancer symptom information.
  • Breast cancer symptom information may include painless lumps, lumps in the breast and armpits, bloody discharge from the nipples, eczema on the nipples, changes in breast skin, nipple depression, changes in breast size, changes in nipple position, and breast pain.
  • the mark may have the shape of a circular (M1) mark in the case of a painless lump, and may have the shape of an asterisk (M2) mark in the case of a breast pain symptom.
  • the user can record the symptoms by using a separate mark on the breast image according to each symptom, and through these images, it is possible to easily determine the opinion on the need for consultation with the medical staff or the opinion on the need for a detailed examination, which will be described later.
  • step (S260) of transmitting to the user terminal a checkup opinion including an opinion on the need for consultation with a medical staff provided based on the health information received from the user terminal, a breast image, and a breast cancer self-examination record, and an opinion on the need for a detailed examination will be performed can
  • a user including a hospital guide information close to the screen of the user terminal, a hospital call connection unit, and a checkup opinion transmission unit for transmitting the examination opinion to a hospital server An interface can be provided.
  • the breast cancer self-examination program and method according to an embodiment of the present invention can effectively prevent and manage breast cancer by linking the self-examination record with a cooperative hospital, and at the same time provide an alternative to the user to reduce psychological anxiety.
  • the effect of the present invention is to help a user perform breast cancer self-examination in a convenient way using a mobile terminal, and provide information on the need for medical treatment or precise diagnosis as a result of the self-examination to help early detection.
  • the present invention interlocks with the camera of the mobile terminal to allow the user to perform breast cancer self-examination by using the breast cancer self-examination sequence audio and the breast cancer screening sequence guide displayed on the recording screen and the breast promotion direction guide implemented in augmented reality. can be done easily.
  • the present invention analyzes a user's breast image using a deep learning model that has learned the breast image, and can provide the user with information on the need for more accurate breast cancer treatment.
  • the breast cancer self-examination method may also be implemented in the form of a recording medium including instructions executable by a computer, such as a program module executed by a computer.
  • Computer-readable media can be any available media that can be accessed by a computer and includes both volatile and nonvolatile media, removable and non-removable media.
  • Computer-readable media may include computer storage media.
  • Computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data.

Landscapes

  • Health & Medical Sciences (AREA)
  • Engineering & Computer Science (AREA)
  • Medical Informatics (AREA)
  • Public Health (AREA)
  • Epidemiology (AREA)
  • Biomedical Technology (AREA)
  • Primary Health Care (AREA)
  • General Health & Medical Sciences (AREA)
  • Data Mining & Analysis (AREA)
  • Pathology (AREA)
  • Databases & Information Systems (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Radiology & Medical Imaging (AREA)
  • Medical Treatment And Welfare Office Work (AREA)
  • Apparatus For Radiation Diagnosis (AREA)

Abstract

A breast cancer self-examination method using a breast cancer self-examination server, according to an embodiment of the present invention, comprises the steps of: receiving health information from a user terminal; transmitting, to the user terminal, breast image capture guide information using augmented reality; receiving, from the user terminal, a breast image captured according to the transmitted guide information; transmitting a breast self-examination record book to the user terminal that has transmitted the breast image; receiving a breast self-examination record recorded on the breast self-examination record book from the user terminal that has received the breast self-examination record book; and transmitting, to the user terminal, an examination opinion, including an opinion on whether a consultation with medical staff is required and an opinion on whether a detailed examination is required, provided on the basis of the health information, the breast image, and a breast cancer self-examination record which are received from the user terminal.

Description

유방암 자가 검진 서버 및 그 방법Breast cancer self-examination server and method therefor
본 발명에 따른 유방암 자가 검진 서버 및 그 방법은 병원에 내원하지 않고 사용자 단말을 이용하여 유방암을 자가 검진하고 진료 필요 여부를 제공하기 위한 유방암 자가 검진 서버 및 그 방법에 관한 것이다.The breast cancer self-examination server and method according to the present invention relates to a breast cancer self-examination server and method for self-examing breast cancer using a user terminal without visiting a hospital and providing whether or not treatment is required.
최근에 암 관련 통계조사에서 유방암은 전체 암 발생의 5위를 차지할 정도로 발병율이 높은 암에 속한다. 유방암 환자는 거의 대부분이 여자이고, 특히 여성의 암 중에서는 (갑상선 암을 제외하고) 발병율이 1위에 달한다. 유방암 환자는 남녀를 합쳐서 연령대별로 보면 40대가 가장 많았고, 50대와 60대 순으로 환자수가 많았다.In recent cancer-related statistical surveys, breast cancer is one of the cancers with a high incidence rate, occupying the fifth place among all cancers. Almost all breast cancer patients are women, and in particular, among female cancers (except for thyroid cancer), the incidence rate is the highest. In terms of age group, the number of breast cancer patients was the highest in their 40s, followed by those in their 50s and 60s.
보통 만 40세 이상 여성은 엑스선(X-ray) 유방 촬상법에 따른 유방암 검진을 받고 있다. 엑스선 유방 촬상의 결과로서 암조직으로 의심되는 조직이 발견되면, 환자는 정밀 진단을 받게 된다. 혹은, 건강진단의 결과로서 환자가 직접 체크하는 환자의 유방에 비정상인 특징(예를 들어, 응어리)을 느끼는지, 또는 의사가 임상, 병력 또는 다른 수단을 통하여 암의 리스크가 비정상으로 높다고 판단되는 상황에 정밀 진단을 수행한다.Usually, women over the age of 40 are screened for breast cancer using X-ray mammography. When a tissue suspected of being cancerous is found as a result of X-ray mammography, the patient is subjected to a precise diagnosis. Or, as a result of the medical examination, whether the patient feels an abnormal characteristic (eg, lump) in the breast of the patient to be checked directly, or the doctor determines that the risk of cancer is abnormally high through clinical, medical history, or other means perform a detailed diagnosis on
다만, 이러한 엑스선을 이용한 유방암 검진 방법은 병원 내원이 필수적으로 요구되며, 번거로움이 따르므로 대부분의 여성들이 주기적으로 유방암 검진을 실시하기 어려운 실정이다.However, this method of breast cancer screening using X-rays requires a hospital visit and is cumbersome, so it is difficult for most women to periodically perform breast cancer screening.
본 발명은 주기적으로 병원에 방문하여 유방암 검진을 실시하는 것의 한계, 자가 검진이 필요하지만, 이 또한 주기적 검진의 어려움, 자가 검진 가이드의 부재, 과거 검진 기록과의 비교 어려움 등이 있으므로 이러한 문제점을 해결하기 위한 유방암 자가 검진 서버 및 그 방법을 제공하고자 한다.The present invention solves these problems because there are limitations in periodically visiting a hospital to conduct breast cancer screening, and self-examination is required, but there are also difficulties in periodic examination, absence of a self-examination guide, and difficulty in comparison with past examination records. An object of the present invention is to provide a breast cancer self-examination server and a method therefor.
또한, 본 발명은 모바일 단말을 이용하여 사용자가 간편한 방법으로 유방암 자가 검진을 수행하고, 자가 검진의 결과로서 진료 필요 또는 정밀 진단 필요에 대한 정보를 제공하여, 유방암의 조기 발견을 돕고자 한다.In addition, an object of the present invention is to assist a user in early detection of breast cancer by performing self-examination of breast cancer in a convenient way by a user using a mobile terminal and providing information on the need for medical treatment or precise diagnosis as a result of the self-examination.
본 발명의 일 실시예에 따른, 유방암 자가 검진 서버를 이용한 유방암 자가 검진 방법에 있어서, 사용자 단말로부터 건강 정보를 수신하는 단계; 상기 사용자 단말로 증강현실을 이용한 유방 이미지 촬영 가이드 정보를 송신하는 단계; 상기 송신된 가이드 정보에 따라 촬영된 유방 이미지를 상기 사용자 단말로부터 수신하는 단계; 상기 유방 이미지를 송신한 사용자 단말로 유방 자가 검진 기록부를 송신하는 단계; 상기 유방 자가 검진 기록부를 수신한 사용자 단말로부터 상기 유방 자가 검진 기록부에 기록된 유방 자가 검진 기록을 수신하는 단계; 및 상기 사용자 단말로부터 수신한 건강 정보, 유방 이미지 및 유방암 자가 검진 기록을 기초로 제공되는 의료진 상담 필요 의견, 정밀 검진 필요 의견을 포함하는 검진 의견을 상기 사용자 단말로 송신하는 단계를 포함하는, 유방암 자가 검진 방법을 제공하고자 한다.According to an embodiment of the present invention, there is provided a breast cancer self-examination method using a breast cancer self-examination server, the method comprising: receiving health information from a user terminal; transmitting breast image capturing guide information using augmented reality to the user terminal; receiving a breast image photographed according to the transmitted guide information from the user terminal; transmitting a breast self-examination record unit to the user terminal that has transmitted the breast image; receiving a breast self-examination record recorded in the breast self-examination record unit from a user terminal that has received the breast self-examination record unit; and transmitting, to the user terminal, a checkup opinion including an opinion on the need for consultation with a medical staff provided based on the health information received from the user terminal, a breast image, and a breast cancer self-examination record, and an opinion on the need for a detailed examination to the user terminal. We want to provide a screening method.
본 실시예에 있어서, 상기 건강 정보는 유방암과 연관된 문진 정보를 포함하는, 유방암 자가 검진 방법을 제공할 수 있다.In this embodiment, the health information may provide a breast cancer self-examination method including questionnaire information related to breast cancer.
본 실시예에 있어서, 상기 유방암과 연관된 문진 정보는 유방암 자가 검진 시기, 월경 여부, 월경일에 기초한 유방 자가 검진 시기, 유방암 가족력 여부, 출산 여부, 30세 이후 첫 출산 여부, 비만 여부, 여성호르몬 투여 여부, 유방 방사선 치료 여부 및 유방 악성 종양 여부 중 적어도 어느 하나를 포함하는, 유방암 자가 검진 방법을 제공할 수 있다.In this embodiment, the breast cancer-related questionnaire information includes breast cancer self-examination time, menstruation, breast self-examination time based on menstruation date, breast cancer family history, childbirth, first childbirth after 30 years of age, obesity, female hormone administration It is possible to provide a breast cancer self-examination method, including at least one of whether or not the breast cancer has been treated with breast radiation, and whether the breast cancer is malignant.
본 실시예에 있어서, 상기 유방 이미지 촬영 가이드 정보는, 유방 촉진 방법을 알려주는 음성 정보 및 이미지 정보를 포함하는, 유방암 자가 검진 방법을 제공할 수 있다.In this embodiment, the breast imaging guide information may provide a breast cancer self-examination method, including voice information and image information indicating a breast palpation method.
본 실시예에 있어서, 상기 음성 정보는 유방 촉진 순서에 따라 출력되는 화면에 매칭되어 출력되는 유방 촉진 방법 안내 음성을 포함하는, 유방암 자가 검진 방법을 제공할 수 있다.In the present embodiment, the breast cancer self-examination method may be provided, wherein the voice information includes a breast palpation method guide voice output by matching a screen output according to the breast palpation order.
본 실시예에 있어서, 상기 이미지 정보는 상기 사용자 단말의 카메라 촬영 화면에 표시되는 유방 배치 가이드 표시 및 유방 촉진 방향 가이드 표시 중 적어도 어느 하나를 포함하는, 유방암 자가 검진 방법을 제공할 수 있다.In this embodiment, the image information may provide a breast cancer self-examination method including at least one of a breast placement guide display and a breast promotion direction guide display displayed on a camera photographing screen of the user terminal.
본 실시예에 있어서, 미리 수집된 유방 이미지 데이터를 기초로 학습된 딥러닝 모델을 통해 카메라 화면 상에 표시되는 사용자의 유방을 인식하고, 상기 인식된 유방 이미지를 기초로 상기 증강현실을 이용한 유방 이미지 촬영 가이드 정보를 제공하되, 상기 미리 수집된 유방 이미지 데이터는 유방의 전체 범위, 유두, 유륜, 쇄골 및 겨드랑이 중 어느 하나로 레이블링된 유방 이미지 데이터인, 유방암 자가 검진 방법을 제공할 수 있다.In this embodiment, the user's breast displayed on the camera screen is recognized through a deep learning model learned based on the previously collected breast image data, and the breast image using the augmented reality based on the recognized breast image It is possible to provide a method for self-examination of breast cancer in which the photographing guide information is provided, but the pre-collected breast image data is breast image data labeled with any one of the full range of the breast, nipples, areolas, clavicle, and armpit.
본 실시예에 있어서, 상기 검진 의견이 의료진 상담 필요 또는 정밀 검진 필요 중 어느 하나인 경우에, 상기 사용자 단말의 화면에 가까운 병원 안내 정보, 병원 통화 연결부, 병원 서버로 검진 의견을 전송하기 위한 검진 의견 전송부를 포함하는 유저 인터페이스가 제공되도록 하는, 유방암 자가 검진 방법을 제공할 수 있다.In this embodiment, when the examination opinion is any one of the need for consultation with a medical team or the need for a detailed examination, the examination opinion for transmitting the examination opinion to the hospital guide information close to the screen of the user terminal, the hospital call connection unit, and the hospital server It is possible to provide a breast cancer self-examination method in which a user interface including a transmission unit is provided.
본 실시예에 있어서, 상기 자가 검진 기록은 상기 유방 이미지 상에 표시되고, 유방암 증상 정보에 따라 다른 형상을 가지는 마크를 포함하는, 유방암 자가 검진 방법을 제공할 수 있다.In this embodiment, the self-examination record may be displayed on the breast image and include a mark having a different shape according to breast cancer symptom information, the breast cancer self-examination method may be provided.
본 실시예에 있어서, 상기 유방암 증상 정보는 통증 없는 멍울, 유방 및 겨드랑이의 덩어리, 유두의 피 섞인 분비물, 젖꼭지의 습진, 유방 피부 변화, 유두 함몰, 유방 크기 변화, 유두 위치 변화 및 유방 통증 중 적어도 어느 하나를 포함하는, 유방암 자가 검진 방법을 제공할 수 있다.In this embodiment, the breast cancer symptom information includes at least one of a painless lump, a lump in the breast and armpit, a bloody discharge from the nipple, eczema of the nipple, a change in breast skin, a nipple depression, a change in breast size, a change in nipple position, and breast pain. It is possible to provide a breast cancer self-examination method, including any one.
본 실시예에 있어서, 상기 사용자 단말로부터 상기 유방 이미지를 수신한 날짜 및 상기 건강 정보에 포함된 월경 정보를 기초로 예정 검진일을 계산하고, 상기 예정 검진일에 대응하여 상기 사용자 단말에 알림을 송신하는, 유방암 자가 검진 방법을 제공할 수 있다.In this embodiment, a scheduled examination date is calculated based on the date the breast image is received from the user terminal and menstrual information included in the health information, and a notification is sent to the user terminal in response to the scheduled examination date It is possible to provide a breast cancer self-examination method.
본 발명의 일 실시예에 따른 유방암 자가 검진 서버에 있어서, 유방암 자가 검진 프로그램이 기록된 메모리; 및 상기 메모리에 기록된 프로그램을 실행하는 프로세서를 포함하고, 상기 프로세서는 상기 프로그램의 실행에 따라, 상기 사용자 단말로 증강현실을 이용한 유방 이미지 촬영 가이드 정보를 송신하고, 상기 송신된 가이드 정보에 따라 촬영된 유방 이미지를 상기 사용자 단말로부터 수신하고, 상기 유방 이미지를 송신한 사용자 단말로 유방 자가 검진 기록부를 송신하고, 상기 유방 자가 검진 기록부를 수신한 사용자 단말로부터 상기 유방 자가 검진 기록부에 기록된 유방 자가 검진 기록을 수신하고, 상기 사용자 단말로부터 수신한 건강 정보, 유방 이미지 및 유방암 자가 검진 기록을 기초로 제공되는 의료진 상담 필요 의견, 정밀 검진 필요 의견을 포함하는 검진 의견을 상기 사용자 단말로 송신하는, 유방암 자가 검진 서버를 제공하고자 한다.In one embodiment of the present invention, there is provided a breast cancer self-examination server, comprising: a memory in which a breast cancer self-examination program is recorded; and a processor executing the program recorded in the memory, wherein the processor transmits breast image photographing guide information using augmented reality to the user terminal according to the execution of the program, and photographing according to the transmitted guide information receiving a breast image from the user terminal, transmitting a breast self-examination record to the user terminal that has transmitted the breast image, and receiving the breast self-examination record recorded in the breast self-examination record from the user terminal receiving the breast self-examination record Breast cancer self that receives the record and transmits to the user terminal a checkup opinion including an opinion on the need for medical staff consultation and an opinion on the need for a detailed examination provided based on the health information received from the user terminal, a breast image, and a breast cancer self-examination record We want to provide a checkup server.
본 발명의 효과는 모바일 단말을 이용하여 사용자가 간편한 방법으로 유방암 자가 검진을 수행하고, 자가 검진의 결과로서 진료 필요 또는 정밀 진단 필요에 대한 정보를 제공하여, 유방암의 조기 발견을 도울 수 있다.According to an effect of the present invention, a user can perform breast cancer self-examination in a convenient way using a mobile terminal, and as a result of the self-examination, information on the need for medical treatment or precise diagnosis can be provided to help early detection of breast cancer.
구체적으로, 본 발명은 모바일 단말의 카메라와 연동하여, 유방암 자가 검진 순서 오디오 및 촬영 화면 상에 표시되는 유방암 검진 순서 가이드 및 증강현실로 구현된 유방 촉진 방향 가이드를 이용하여, 사용자가 유방암 자가 검진을 용이하게 수행하도록 할 수 있다.Specifically, the present invention interlocks with the camera of the mobile terminal to allow the user to perform breast cancer self-examination by using the breast cancer self-examination sequence audio and the breast cancer screening sequence guide displayed on the recording screen and the breast promotion direction guide implemented in augmented reality. can be done easily.
또한, 본 발명은 유방 이미지를 학습한 딥러닝 모델을 이용하여 사용자의 유방 이미지를 분석하여, 사용자에게 보다 정확도 높은 유방암 진료 필요성에 대한 정보를 제공할 수 있다.In addition, the present invention analyzes a user's breast image using a deep learning model that has learned the breast image, and can provide the user with information on the need for more accurate breast cancer treatment.
도 1은 본 발명의 일 실시예에 따른 유방암 자가 검진 서버의 구성을 보여주는 구성도이다.1 is a configuration diagram showing the configuration of a breast cancer self-examination server according to an embodiment of the present invention.
도 2는 본 발명의 일 실시예에 따른 유방암 자가 검진 프로그램의 순서도이다.2 is a flowchart of a breast cancer self-examination program according to an embodiment of the present invention.
도 3은 본 발명의 일 실시예에 따른 유방암 자가 검진 방법을 보여주는 순서도이다.3 is a flowchart illustrating a breast cancer self-examination method according to an embodiment of the present invention.
도 4는 본 발명의 일 실시예에 따른 유방암 자가 검진 프로그램 로그인 화면 및 메인 화면 인터페이스를 보여주는 도면이다.4 is a diagram illustrating a breast cancer self-examination program login screen and a main screen interface according to an embodiment of the present invention.
도 5는 본발명의 일 실시예에 따른 건강 정보 입력 항목의 예시를 보여주는 도면이다.5 is a diagram illustrating an example of a health information input item according to an embodiment of the present invention.
도 6 내지 도 9는 본 발명의 일 실시예에 따른 유방 이미지 촬영 가이드 유저 인터페이스를 보여주는 도면이다.6 to 9 are diagrams illustrating a breast image capturing guide user interface according to an embodiment of the present invention.
도 10은 본 발명의 일 실시예에 따른 사용자 자가 검진 기록일지를 보여주는 도면이다.10 is a view showing a user self-examination record log according to an embodiment of the present invention.
도 11은 본 발명의 일 실시예에 따른 유방암 자가 검진 기록부에 유방 이미지 상에 표시되는 마크를 보여주는 도면이다.11 is a view showing a mark displayed on a breast image in a breast cancer self-examination recorder according to an embodiment of the present invention.
아래에서는 첨부한 도면을 참조하여, 본 발명이 속하는 기술 분야에서 통상의 지식을 가진 자가 용이하게 실시할 수 있도록 본 발명의 실시예를 상세히 설명한다. 그러나, 본 발명은 여러 가지 상이한 형태로 구현될 수 있으며 여기에서 설명하는 실시예에 한정되지 않는다. 그리고, 도면에서 본 발명을 명확하게 설명하기 위해서 설명과 관계없는 부분은 생략하였으며, 명세서 전체를 통하여 유사한 부분에 대해서는 유사한 도면 부호를 붙였다.Hereinafter, with reference to the accompanying drawings, embodiments of the present invention will be described in detail so that those of ordinary skill in the art can easily carry out the present invention. However, the present invention may be embodied in many different forms and is not limited to the embodiments described herein. And, in order to clearly explain the present invention in the drawings, parts irrelevant to the description are omitted, and similar reference numerals are attached to similar parts throughout the specification.
명세서 전체에서, 어떤 부분이 다른 부분과 "연결"되어 있다고 할 때, 이는 "직접적으로 연결"되어 있는 경우뿐만 아니라, 그 중간에 다른 소자를 사이에 두고 "전기적으로 연결"되어 있는 경우도 포함한다. 또한, 어떤 부분이 어떤 구성요소를 "포함"한다고 할 때, 이는 특별히 반대되는 기재가 없는 한 다른 구성요소를 제외하는 것이 아니라 다른 구성요소를 더 포함할 수 있는 것을 의미한다.Throughout the specification, when a part is "connected" to another part, this includes not only the case of being "directly connected" but also the case of being "electrically connected" with another element interposed therebetween. . In addition, when a part "includes" a certain component, this means that other components may be further included, rather than excluding other components, unless otherwise stated.
이하에서는 본 발명의 일 실시예에 따른 유방암 자가 검진 서버에 대하여 설명하기로 한다.Hereinafter, a breast cancer self-examination server according to an embodiment of the present invention will be described.
도 1은 본 발명의 일 실시예에 따른 유방암 자가 검진 서버(100)의 구성을 보여주는 구성도이다. 도 2는 본 발명의 일 실시예에 따른 유방암 자가 검진 프로그램의 순서도이다.1 is a configuration diagram showing the configuration of a breast cancer self-examination server 100 according to an embodiment of the present invention. 2 is a flowchart of a breast cancer self-examination program according to an embodiment of the present invention.
도 1 및 도 2를 참조하면 본 발명의 일 실시예에 따른 유방암 자가 검진 서버(100)는 사용자 단말의 유방암 자가 검진 프로그램의 실행에 따라 카메라를 활용하여 사용자가 스스로 유방 내 존재하는 멍울, 종양 등의 위치나 크기 등을 파악할 수 있도록 딥러닝(Deep Learning)과 증강현실(AR: Augmented Reality)기술을 도입한 영상 안내를 실시하고, 이에 매칭되는 스피커를 통한 음성 안내를 동반하는 유방암 자가 검진을 실행하기 위한 서버이다.1 and 2 , the breast cancer self-examination server 100 according to an embodiment of the present invention utilizes a camera according to the execution of the breast cancer self-examination program of the user terminal to allow the user to voluntarily exist in the breast, such as lumps, tumors, etc. In order to determine the location and size of the breast cancer, self-examination of breast cancer accompanied by voice guidance through a speaker that matches the video guidance using deep learning and augmented reality (AR) technology is implemented. server to do it.
본 발명의 일 실시예에 따른 유방암 자가 검진 서버(100)는 사용자 단말간의 통신을 통해 건강 정보 수신, 유방암 자가 검진 이미지 촬영 및 기록 수신, 건강 정보 점수와 유방암 자가 검진 기록 점수를 기초로 진료 필요 여부 판정 및 병원 연계의 과정을 수행할 수 있으며 자세한 방법에 대해서는 후술하기로 한다.The breast cancer self-examination server 100 according to an embodiment of the present invention receives health information through communication between user terminals, photographing and recording breast cancer self-examination images, and whether treatment is necessary based on health information scores and breast cancer self-examination record scores The process of judgment and hospital linkage can be performed, and the detailed method will be described later.
한편 유방암 자가 검진 서버(100)는 통신 모듈(110), 메모리(120), 프로세서(130), 데이터베이스(DB: 140)를 포함할 수 있다.Meanwhile, the breast cancer self-examination server 100 may include a communication module 110 , a memory 120 , a processor 130 , and a database (DB: 140 ).
통신 모듈(110)은 통신망과 연동하여 유방암 자가 검진 서버(100)에 통신 인터페이스를 제공하는데, 사용자 단말과 데이터를 송수신하는 역할을 수행할 수 있다. 여기서, 통신 모듈(110)은 다른 네트워크 장치와 유무선 연결을 통해 제어 신호 또는 데이터 신호와 같은 신호를 송수신하기 위해 필요한 하드웨어 및 소프트웨어를 포함하는 장치일 수 있다. The communication module 110 provides a communication interface to the breast cancer self-examination server 100 by interworking with a communication network, and may serve to transmit and receive data to and from a user terminal. Here, the communication module 110 may be a device including hardware and software necessary for transmitting and receiving signals such as control signals or data signals through wired/wireless connection with other network devices.
메모리(120)는 유방암 자가 검진 방법 프로그램이 기록된 것일 수 있다. 또한, 메모리(120)는 프로세서(130)가 처리하는 데이터를 일시적 또는 영구적으로 저장하는 기능을 수행할 수 있다. 여기서, 메모리(120)는 휘발성 저장 매체(volatile storage media) 또는 비휘발성 저장 매체(non-volatile storage media)를 포함할 수 있으나, 본 발명의 범위가 이에 한정되는 것은 아니다.The memory 120 may record a breast cancer self-examination method program. In addition, the memory 120 may perform a function of temporarily or permanently storing data processed by the processor 130 . Here, the memory 120 may include a volatile storage medium or a non-volatile storage medium, but the scope of the present invention is not limited thereto.
프로세서(130)는 유방암 자가 검진 서버(100)에서 유방암 자가 검진 방법 프로그램이 수행하는 전체 과정을 제어할 수 있다. 프로세서(130)가 수행하는 과정의 각 단계에 대해서는 도 3 내지 도 11을 참조하여 후술하기로 한다.The processor 130 may control the entire process performed by the breast cancer self-examination method program in the breast cancer self-examination server 100 . Each step of the process performed by the processor 130 will be described later with reference to FIGS. 3 to 11 .
여기서, 프로세서(130)는 프로세서(processor)와 같이 데이터를 처리할 수 있는 모든 종류의 장치를 포함할 수 있다. 여기서, '프로세서(processor)'는, 예를 들어 프로그램 내에 포함된 코드 또는 명령으로 표현된 기능을 수행하기 위해 물리적으로 구조화된 회로를 갖는, 하드웨어에 내장된 데이터 처리 장치를 의미할 수 있다. 이와 같이 하드웨어에 내장된 데이터 처리 장치의 일 예로서, 마이크로프로세서(microprocessor), 중앙처리장치(central processing unit: CPU), 프로세서 코어(processor core), 멀티프로세서(multiprocessor), ASIC(application-specific integrated circuit), FPGA(field programmable gate array) 등의 처리 장치를 망라할 수 있으나, 본 발명의 범위가 이에 한정되는 것은 아니다.Here, the processor 130 may include all kinds of devices capable of processing data, such as a processor. Here, the 'processor' may refer to a data processing device embedded in hardware, for example, having a physically structured circuit to perform a function expressed as a code or an instruction included in a program. As an example of the data processing device embedded in the hardware as described above, a microprocessor, a central processing unit (CPU), a processor core, a multiprocessor, an application-specific integrated (ASIC) circuit) and a processing device such as a field programmable gate array (FPGA), but the scope of the present invention is not limited thereto.
데이터베이스(140)는 후술할 머신러닝에 의해 학습되는 유방 이미지 데이터를 포함하여, 유방암 자가 검진 프로그램이 실행되는데 필요한 데이터가 저장된 것일 수 있다.The database 140 may store data necessary for executing a breast cancer self-examination program, including breast image data learned by machine learning to be described later.
한편, 본 발명에서 '단말'은 휴대성 및 이동성이 보장된 무선 통신 장치일 수 있으며, 예를 들어 스마트 폰, 태블릿 PC 또는 노트북 등과 같은 모든 종류의 핸드헬드(Handheld) 기반의 무선 통신 장치일 수 있다.Meanwhile, in the present invention, a 'terminal' may be a wireless communication device with guaranteed portability and mobility, for example, any type of handheld-based wireless communication device such as a smart phone, tablet PC, or notebook computer. have.
또한, '단말'은 네트워크를 통해 다른 단말 또는 서버 등에 접속할 수 있는 PC 등의 유선 통신 장치인 것도 가능하다.In addition, the 'terminal' may be a wired communication device such as a PC that can be connected to another terminal or a server through a network.
또한, 네트워크는 단말들 및 서버들과 같은 각각의 노드 상호 간에 정보 교환이 가능한 연결 구조를 의미하는 것으로, 근거리 통신망(LAN: Local Area Network), 광역 통신망(WAN: Wide Area Network), 인터넷 (WWW: World Wide Web), 유무선 데이터 통신망, 전화망, 유무선 텔레비전 통신망 등을 포함한다.In addition, the network refers to a connection structure capable of exchanging information between each node, such as terminals and servers, and includes a local area network (LAN), a wide area network (WAN), and the Internet (WWW). : World Wide Web), wired and wireless data networks, telephone networks, and wired and wireless television networks.
무선 데이터 통신망의 일례에는 3G, 4G, 5G, 3GPP(3rd Generation Partnership Project), LTE(Long Term Evolution), WIMAX(World Interoperability for Microwave Access), 와이파이(Wi-Fi), 블루투스 통신, 적외선 통신, 초음파 통신, 가시광 통신(VLC: Visible Light Communication), 라이파이(LiFi) 등이 포함되나 이에 한정되지는 않는다.Examples of wireless data communication networks include 3G, 4G, 5G, 3rd Generation Partnership Project (3GPP), Long Term Evolution (LTE), World Interoperability for Microwave Access (WIMAX), Wi-Fi, Bluetooth communication, infrared communication, ultrasound Communication, Visible Light Communication (VLC), LiFi, etc. are included, but are not limited thereto.
이하에서는 본 발명의 일 실시예에 따른 유방암 자가 검진 방법에 대하여 설명하기로 한다.Hereinafter, a breast cancer self-examination method according to an embodiment of the present invention will be described.
도 3은 본 발명의 일 실시예에 따른 유방암 자가 검진 방법을 보여주는 순서도이다. 도 4는 본 발명의 일 실시예에 따른 유방암 자가 검진 프로그램 로그인 화면 및 메인 화면 인터페이스를 보여주는 도면이다. 도 5는 본발명의 일 실시예에 따른 건강 정보 입력 항목의 예시를 보여주는 도면이다. 3 is a flowchart illustrating a breast cancer self-examination method according to an embodiment of the present invention. 4 is a diagram illustrating a breast cancer self-examination program login screen and a main screen interface according to an embodiment of the present invention. 5 is a diagram illustrating an example of a health information input item according to an embodiment of the present invention.
도 3을 참조하면, 본 발명의 일 실시예에 따른 유방암 자가 검진 서버(100)를 이용한 유방암 자가 검진 방법은 먼저, 사용자 단말로부터 건강 정보를 수신하는 단계(S210)가 수행될 수 있다.Referring to FIG. 3 , in the breast cancer self-examination method using the breast cancer self-examination server 100 according to an embodiment of the present invention, first, receiving health information from a user terminal ( S210 ) may be performed.
도 4를 참조하면, 사용자는 단말 상의 유방암 자가 검진 프로그램의 유저 인터페이스에서 로그인을 수행할 수 있다(도 4의 (a) 참조). 사용자가 로그인하면 유저 인터페이스는 마지막 유방암 자가 검진일로부터 현재까지 경과된 일수를 표시하고, 검사 시작 버튼, 전문가 상담 버튼, 기록보기 버튼, 유방 정보를 제공하는 바로알기 버튼을 출력할 수 있다(도 4의 (b) 참조). 유방 정보는 유방암의 정의, 통계 및 원인과 증상, 유방암 자가검진의 필요성, 유방암 예방방법을 포함할 수 있다.Referring to FIG. 4 , the user may log in through the user interface of the breast cancer self-examination program on the terminal (see FIG. 4A ). When the user logs in, the user interface may display the number of days elapsed from the date of the last breast cancer self-examination to the present, and output an examination start button, expert consultation button, record view button, and a quick know button providing breast information (FIG. 4) of (b)). Breast information may include the definition of breast cancer, statistics, causes and symptoms, the need for breast cancer self-examination, and methods for preventing breast cancer.
도 5를 참조하면, 사용자가 유방암 자가 검진 검사를 시작하면 먼저 사용자는 단말의 유방암 자가 검진 프로그램에 건강 정보를 입력할 수 있다. 한편, 건강 정보는 유방암 관련 문진 정보를 포함할 수 있다. 유방암 관련 문진 정보는 유방암 자가 검진 시기, 월경 여부, 월경일에 기초한 유방 자가 검진 시기, 유방암 가족력 여부, 출산 여부, 30세 이후 첫 출산 여부, 비만 여부, 여성호르몬 투여 여부, 유방 방사선 치료 여부 및 유방 악성 종양 여부 중 적어도 어느 하나를 포함할 수 있다.Referring to FIG. 5 , when the user starts the breast cancer self-examination test, the user may first input health information into the breast cancer self-examination program of the terminal. Meanwhile, the health information may include breast cancer-related questionnaire information. Breast cancer-related questionnaire information includes breast cancer self-examination time, menstruation status, breast self-examination time based on menstruation date, family history of breast cancer, childbirth, first childbirth after 30 years of age, obesity, female hormone administration, breast radiation treatment, and breast It may include at least one of malignant tumors.
도 6 내지 도 9는 본 발명의 일 실시예에 따른 유방 이미지 촬영 가이드 유저 인터페이스를 보여주는 도면이다. 6 to 9 are diagrams illustrating a breast image capturing guide user interface according to an embodiment of the present invention.
다음으로 도 6 내지 도 9를 참조하면, 사용자 단말로 증강현실을 이용한 유방 이미지 촬영 가이드 정보를 송신하는 단계(S220)가 수행될 수 있다.Next, referring to FIGS. 6 to 9 , a step ( S220 ) of transmitting breast image capturing guide information using augmented reality to the user terminal may be performed.
유방 이미지 촬영 가이드 정보는, 유방 촉진 방법을 알려주는 음성 정보와 이미지 정보를 포함할 수 있다. 이미지 정보는 사용자 단말의 카메라 촬영 화면 상에 표시되는 유방 배치 가이드 표시 및 유방 촉진 방향 가이드 표시 중 적어도 어느 하나를 포함할 수 있다.The breast imaging guide information may include voice information and image information indicating a breast palpation method. The image information may include at least one of a breast placement guide display and a breast promotion direction guide display displayed on a camera photographing screen of the user terminal.
도 6을 참조하면, 사용자는 단말의 인터페이스에서 측정하기 버튼을 클릭하여 유방 자가 검진을 시작하거나, 사용법안내 버튼을 클릭하여 사용법 안내를 받을 수 있다(도 6의 (a) 참조). 측정하기 버튼을 클릭하면 단말의 카메라 촬영 화면에서 증강현실로 표시되는 라인 상에 사용자의 가슴을 배치시키도록 안내되고 측정시작 버튼이 제공될 수 있다(도 6의 (b) 참조). 측정이 완료되면 유방 촬영 이미지를 저장하기 위한 완료하기 버튼, 검진 기록을 작성하기 위한 일지쓰기 버튼이 출력될 수 있다(도 6의 (c) 참조).Referring to FIG. 6 , the user may click the measure button on the interface of the terminal to start the breast self-examination, or click the usage guide button to receive a usage guide (refer to FIG. 6(a)). When the measurement button is clicked, the user is guided to place the chest of the user on the line displayed in augmented reality on the camera shooting screen of the terminal, and a measurement start button may be provided (see (b) of FIG. 6). When the measurement is completed, a finish button for storing the mammogram image and a journal entry button for writing a checkup record may be output (refer to FIG. 6(c) ).
유방 촉진 방법을 알려주는 음성 정보는 아래와 같은 순서로 음성과 참고 안내 화면이 매칭되도록 출력될 수 있다.The voice information informing the breast palpation method may be output so that the voice and the reference guidance screen are matched in the following order.
먼저, 촉진 시작을 알리는 안내 음성이 출력될 수 있다. 다음으로 사진 및/또는 영상 촬영을 위해 사용자 단말을 책상, 화장대, 침대 등에 수평으로 고정시키기 위한 안내 음성이 출력될 수 있다. 여기서 도 6의 (a)에 개시된 안내 화면이 출력될 수 있다. First, a guide voice indicating the start of promotion may be output. Next, a guide voice for horizontally fixing the user terminal to a desk, a dressing table, a bed, etc. for taking a photo and/or an image may be output. Here, the guidance screen shown in FIG. 6A may be output.
그리고 사용자 단말의 카메라 촬영 화면 상에 사용자의 양쪽 어깨부터 배꼽까지 나타나도록 카메라 초점거리 조절 안내 음성이 출력될 수 있다. 여기서 도 6의 (b)에 개시된 안내 화면이 출력될 수 있다.In addition, a camera focal length adjustment guide voice may be output to appear from both shoulders of the user to the navel on the camera photographing screen of the user terminal. Here, the guidance screen shown in FIG. 6B may be output.
초점거리조절이 완료되면, 사용자 단말에서 카메라 화면에 나타난 사용자의 유방 모양이나 윤곽의 변화를 관찰하라는 안내 음성이 출력될 수 있다. 이때, 관찰 방법으로서, 양팔을 내려놓은 상태에서 유방을 관찰하는 방법(도 6의 (c) 참조), 양손을 머리 뒤로 깍지 낀 상태에서 유방을 관찰하는 방법(도 6의 (d) 참조), 양손을 허리에 짚은 상태에서 유방을 관찰하는 방법(도 6의 (e) 참조)에 대한 안내 음성 및 안내 화면이 일시적으로 출력될 수 있다.When the focal length adjustment is completed, a guide voice instructing the user to observe a change in the shape or contour of the user's breast displayed on the camera screen may be output from the user terminal. At this time, as an observation method, a method of observing the breast with both arms down (see Fig. 6 (c)), a method of observing the breast with both hands interlaced behind the head (see Fig. 6 (d)), A voice guidance and a guidance screen for a method of observing the breast with both hands on the waist (refer to FIG. 6(e)) may be temporarily output.
다음으로, 유방 촉진 방법에 있어서 사용자가 서있거나 앉은 상태로 유방 자가 검진을 실시하는 경우에는 아래와 같은 순서가 진행될 수 있다.Next, in the breast palpation method, when the user performs the breast self-examination in a standing or sitting state, the following procedure may be performed.
먼저 서있거나, 앉은 상태에서 화면에 표시된 원에 한쪽 유방을 맞추도록 안내 음성 출력 및 화면에서 증강현실로 구현되는 유방 인식 표시 안에 사용자의 유방을 배치시키도록 안내 화면이 출력될 수 있다 (도 8의 (a) 참조). 사용자가 유방 인식 표시 안에 사용자의 유방을 배치시키면 사용자 단말에서 이를 인식하고 다음 화면으로 전환될 수 있다.First, a guidance voice output to align one breast with the circle displayed on the screen in a standing or sitting state, and a guidance screen to place the user's breast in the breast recognition display implemented in augmented reality on the screen may be output (FIG. 8) see (a)). When the user places the user's breast in the breast recognition display, the user terminal may recognize it and switch to the next screen.
이 때, 검진하려는 유방쪽 팔을 머리 위로 들어 올리고 반대편 손의 2, 3, 4번째 손가락 첫마디 바닥면을 이용해 촉진하도록 안내하는 음성 및 화면이 출력될 수 있다(도 8의 (b) 참조). 그리고 유방 주위 바깥쪽 상단 부위에서 원을 그려가며 안쪽으로 유방의 위, 아래, 부위와 겨드랑이 밑에서부터 검진하도록 카메라 촬영 화면에 증강현실로 구현되는 유방 촉진 방향 가이드 표시가 출력될 수 있다(도 8의 (c) 참조). 그리고, 유두 주변을 작은 원을 그리며 만져보고, 유두의 위 아래와 양옆에서 안쪽으로 짜보아서 비정상적인 분비물이 있는지 확인하는 방법에 대한 안내 음성 및 화면이 출력될 수 있다(도 8의 (d) 참조).At this time, a voice and a screen for guiding to raise the arm on the side of the breast to be examined over the head and palpate it using the bottom surface of the first joint of the 2nd, 3rd, and 4th fingers of the opposite hand may be output (refer to (b) of FIG. 8). In addition, a breast promotion direction guide display implemented in augmented reality may be output on the camera shooting screen to draw a circle from the outer upper part around the breast and examine the breast from the top, bottom, part, and under the armpit inward (FIG. 8). see (c)). Then, a small circle around the nipple is touched, and a voice guidance and a screen on how to check if there is an abnormal secretion by squeezing inward from the top and bottom of the nipple and both sides may be output (see (d) of FIG. 8).
추가적으로, 사용자가 누운 자세로 자세를 바꾸어서 촉진을 실시하는 경우에는 아래와 같은 순서가 진행될 수 있다.Additionally, when the user changes his or her posture to a supine position and promotes it, the following procedure may be performed.
사용자가 누운 상태에서 화면에 표시된 원에 한쪽 유방을 맞추도록 안내하는 음성 및 화면에서 증강현실로 구현되는 유방 인식 표시 안에 사용자의 유방을 배치하도록 안내하는 화면이 출력될 수 있다(도 9의 (a) 참조). 사용자가 유방 인식 표시 안에 유방을 배치시키면 다음 화면으로 전환될 수 있다.A voice guiding the user to align one breast with the circle displayed on the screen in a lying state and a screen guiding the user to place the user's breast in the breast recognition display implemented in augmented reality on the screen may be output (FIG. 9 (a) ) Reference). When the user places the breast in the breast recognition display, it may be switched to the next screen.
다음 화면에서, 사용자가 누워서 검사하는 유방이 있는 쪽의 팔을 위로 올리고 반대편 손으로 촉진하도록 안내하는 음성 및 화면이 출력될 수 있다(도 9의 (b) 참조).In the next screen, a voice and a screen guiding the user to lie down and raise the arm on the side with the breast to be examined and to facilitate it with the opposite hand may be output (refer to FIG. 9(b) ).
이 때, 검진하려는 유방쪽 팔을 머리 위로 들어 올리고 반대편 손의 2, 3, 4번째 손가락 첫마디 바닥면을 이용해 촉진하도록 안내하는 음성 및 화면이 출력될 수 있다(도 9의 (c) 참조). 그리고 유방 주위 바깥쪽 상단 부위에서 원을 그려가며 안쪽으로 유방의 위, 아래, 부위와 겨드랑이 밑에서부터 검진하도록 화면 상에 증강현실로 구현되는 유방 촉진 방향 가이드 표시가 출력될 수 있다(도 9의 (d) 참조). 그리고, 유두 주변을 작은 원을 그리며 만져보고, 유두의 위 아래와 양옆에서 안쪽으로 짜보아서 비정상적인 분비물이 있는지 확인하는 방법에 대한 안내 음성 및 화면이 출력될 수 있다(도 9의 (e) 참조).At this time, a voice and a screen for guiding to raise the arm on the side of the breast to be examined over the head and palpate it using the bottom surface of the first joint of the 2nd, 3rd, and 4th fingers of the opposite hand may be output (refer to FIG. 9(c)). In addition, a breast promotion direction guide display implemented in augmented reality may be output on the screen to draw a circle from the outer upper part around the breast and check from the upper, lower, and armpit areas inward (see FIG. 9 ( see d)). Then, a small circle around the nipple and touch it, squeeze the nipple from above and below both sides to the inside, so that a voice guidance and a screen on how to check if there is an abnormal secretion may be output (refer to FIG. 9(e)).
촉진을 마친 후에는 촉진 영상의 저장 및 설문지 작성을 요청하는 안내 음성이 출력될 수 있다.After the promotion is completed, a guide voice requesting to store the promotion image and fill out a questionnaire may be output.
위와 같은 유방암 자가 검진 안내 오디오, 안내 화면 및 증강현실로 구현되는 유방 배치 가이드 표시와 유방 촉진 방향 가이드 표시에 따라 사용자는 유방에 띄워진 표식을 따라 촉진할 수 있으며, 유방암과 자가검진법에 대한 올바른 정보를 쉽게 익히고, 자신의 유방에 맞는 방법으로 자가검진을 할 수 있다.According to the above breast cancer self-examination guide audio, guidance screen, and augmented reality display of breast placement guide and breast promotion direction guide display, the user can follow the mark on the breast to promote it, and correct information about breast cancer and self-examination methods. You can easily learn information and do self-examination in a way that suits your breast.
한편, 증강현실을 이용한 유방 이미지 촬영 가이드 정보는, 미리 수집된 유방 이미지 데이터를 기초로 학습된 딥러닝 모델을 통해 카메라 화면 상에 표시되는 사용자의 유방을 인식할 수 있다. 이렇게 인식된 유방에 인접하게 증강현실로 구현되는 유방 이미지 촬영 및 촉진 가이드 표시가 제공될 수 있다. 예를 들어, 상술한 증강현실로 구현되는 유방 배치 가이드 표시와 유방 촉진 방향 가이드 표시가 유방 이미지 촬영 가이드 정보에 포함될 수 있다. 여기서, 미리 수집된 유방 이미지 데이터는 유방의 전체 범위, 유두, 유륜, 쇄골 및 겨드랑이 중 어느 하나로 레이블링된 유방 이미지 데이터일 수 있다.Meanwhile, the breast image capturing guide information using augmented reality may recognize the user's breast displayed on the camera screen through a deep learning model learned based on pre-collected breast image data. Adjacent to the recognized breast in this way, a breast image capturing and facilitation guide display implemented in augmented reality may be provided. For example, the breast placement guide display and the breast promotion direction guide display implemented by the above-described augmented reality may be included in the breast image capturing guide information. Here, the pre-collected breast image data may be breast image data labeled with any one of the full range of the breast, nipple, areola, clavicle, and armpit.
딥러닝 모델은 유방 이미지 상에서 유방 및 기타 유방암 관련 영역을 자동으로 분할 및 추출하고, 그 외 영역을 벗어나는 오검출을 제거할 수 있다. 딥러닝 모델은 유방 및 기타 영역에 대한 관심영역(RoI: Region of Interest) 확보, 분할 및 추출에 특화된 디컨벌루션 네트워크(Deconvolution Network), U-Net 등 딥러닝 모델을 사용하여 유방 이미지를 학습할 수 있다. 또한, 딥러닝 모델은 유효성을 검토하기 위해 학습에 참여하지 않은 데이터를 이용하여 유효성을 검증할 수 있다. 또한, 통계 기법 검증을 통한 이미지 추출 정확도를 검증할 수 있다. 학습된 딥러닝 모델의 Over Fitting 여부를 포함한 객관적인 일반화 성능 검증을 위해 K-Fold 교차검증 수행할 수 있다. 검출 모델은 Sensitivity 및 FPs/image 등을 통해 모델의 성능을 검증하고 평가할 수 있다. The deep learning model can automatically segment and extract the breast and other breast cancer-related areas on the breast image, and remove false detections outside the other areas. Deep learning models can learn breast images using deep learning models such as Deconvolution Network and U-Net specialized in securing, segmenting and extracting Region of Interest (RoI) for breast and other regions. have. In addition, deep learning models can be validated using data that did not participate in training to check their validity. In addition, image extraction accuracy can be verified through statistical technique verification. K-fold cross-validation can be performed to verify objective generalization performance including overfitting of the trained deep learning model. The detection model can verify and evaluate the performance of the model through Sensitivity and FPs/image.
이로써, 본 발명의 일 실시예에 따른 유방 자가 검진 방법은 딥러닝 모델을 이용하여 데이터 전처리, 딥러닝 모델 학습/테스트/검증을 수행하고, 이러한 딥러닝 모델을 통해 사용자의 유방 이미지를 정확하게 인식하여 증강현실 유방 이미지 촬영 가이드 정보를 화면 상에 출력할 수 있다.Thus, in the breast self-examination method according to an embodiment of the present invention, data preprocessing and deep learning model learning/testing/verification are performed using a deep learning model, and the user's breast image is accurately recognized through the deep learning model. Augmented reality breast image shooting guide information may be output on the screen.
다음으로, 송신된 가이드 정보에 따라 촬영된 유방 이미지를 사용자 단말로부터 수신하는 단계(S230)가 수행될 수 있다.Next, a step (S230) of receiving the captured breast image from the user terminal according to the transmitted guide information may be performed.
도 10은 본 발명의 일 실시예에 따른 사용자 자가 검진 기록일지를 보여주는 도면이다. 10 is a view showing a user self-examination record log according to an embodiment of the present invention.
다음으로 도 10을 참조하면, 유방 이미지를 송신한 사용자 단말로 유방 자가 검진 기록부를 송신하는 단계(S240)가 수행될 수 있다. 사용자는 유방암 자가 검진 후 기록부를 작성하고, 유방암 자가 검진 일정 알림을 설정하여 스스로 자신의 유방 데이터를 구축하고 자가검진이 습관이 되도록 할 수 있다. 또한 유방암 자가 검진 서버(100)는 사용자 단말로부터 유방 이미지를 수신한 날짜 및 건강 정보에 포함된 월경 정보를 기초로 예정 검진일을 계산하고, 예정 검진일에 대응하여 사용자 단말에 알림을 송신할 수 있다. 또한, 사용자 단말의 유저 인터페이스 상에 날짜를 입력(도 9의 (a) 참조)하고, 검진 일지를 쓸 수 있는 화면(도 9의 (b), (c) 참조)이 출력될 수 있다. 사용자는 여기에 검진 기록 일지를 기재할 수 있다.Next, referring to FIG. 10 , a step S240 of transmitting the breast self-examination record to the user terminal that has transmitted the breast image may be performed. The user can create a logbook after the breast cancer self-examination and set a reminder for the breast cancer self-exam schedule to build his/her own breast data and make self-examination a habit. In addition, the breast cancer self-examination server 100 may calculate the scheduled examination date based on the date of receiving the breast image from the user terminal and menstruation information included in the health information, and transmit a notification to the user terminal in response to the scheduled examination date. have. In addition, a date may be input on the user interface of the user terminal (refer to (a) of FIG. 9), and a screen (refer to (b) and (c) of FIG. 9) for writing a checkup diary may be output. The user may record the examination log here.
도 11은 본 발명의 일 실시예에 따른 유방암 자가 검진 기록부에 유방 이미지 상에 표시되는 마크를 보여주는 도면이다.11 is a view showing a mark displayed on a breast image in a breast cancer self-examination recorder according to an embodiment of the present invention.
다음으로 도11을 참조하면, 유방 자가 검진 기록부를 수신한 사용자 단말로부터 유방 자가 검진 기록부에 기록된 유방 자가 검진 기록을 수신하는 단계(S250)가 수행될 수 있다.Next, referring to FIG. 11 , receiving the breast self-examination record recorded in the breast self-examination recorder from the user terminal receiving the breast self-examination record ( S250 ) may be performed.
여기서, 자가 검진 기록은 유방 이미지 상에 표시되고, 유방암 증상 정보에 따라 다른 형상을 가지는 마크를 포함할 수 있다. 유방암 증상 정보는 통증 없는 멍울, 유방 및 겨드랑이의 덩어리, 유두의 피 섞인 분비물, 젖꼭지의 습진, 유방 피부 변화, 유두 함몰, 유방 크기 변화, 유두 위치 변화 및 유방 통증을 포함할 수 있다. 일 예로 마크는 통증 없는 멍울 증상의 경우 원형(M1) 마크의 형상을 가질 수 있고, 유방 통증 증상의 경우 별표(M2) 마크의 형상을 가질 수 있다. Here, the self-examination record may be displayed on the breast image and include marks having different shapes according to breast cancer symptom information. Breast cancer symptom information may include painless lumps, lumps in the breast and armpits, bloody discharge from the nipples, eczema on the nipples, changes in breast skin, nipple depression, changes in breast size, changes in nipple position, and breast pain. For example, the mark may have the shape of a circular (M1) mark in the case of a painless lump, and may have the shape of an asterisk (M2) mark in the case of a breast pain symptom.
이로써, 사용자는 각 증상에 따라 유방 이미지에 별개의 마크를 사용하여 증상을 기록할 수 있으며, 이러한 이미지를 통해 후술할 의료진 상담 필요 의견이나, 정밀 검진 필요 의견에 대한 판단을 용이하게 할 수 있다.Accordingly, the user can record the symptoms by using a separate mark on the breast image according to each symptom, and through these images, it is possible to easily determine the opinion on the need for consultation with the medical staff or the opinion on the need for a detailed examination, which will be described later.
다음으로 사용자 단말로부터 수신한 건강 정보, 유방 이미지 및 유방암 자가 검진 기록을 기초로 제공되는 의료진 상담 필요 의견, 정밀 검진 필요 의견을 포함하는 검진 의견을 상기 사용자 단말로 송신하는 단계(S260)가 수행될 수 있다.Next, the step (S260) of transmitting to the user terminal a checkup opinion including an opinion on the need for consultation with a medical staff provided based on the health information received from the user terminal, a breast image, and a breast cancer self-examination record, and an opinion on the need for a detailed examination will be performed can
여기서, 검진 의견이 의료진 상담 필요 또는 정밀 검진 필요 중 어느 하나인 경우에, 상기 사용자 단말의 화면에 가까운 병원 안내 정보, 병원 통화 연결부, 병원 서버로 검진 의견을 전송하기 위한 검진 의견 전송부를 포함하는 유저 인터페이스가 제공되도록 할 수 있다.Here, when the examination opinion is any one of the need for consultation with a medical staff or the need for a detailed examination, a user including a hospital guide information close to the screen of the user terminal, a hospital call connection unit, and a checkup opinion transmission unit for transmitting the examination opinion to a hospital server An interface can be provided.
이로써, 본 발명의 일 실시예에 따른 유방암 자가 검진 프로그램 및 방법은 자가검진기록을 협력 병원과 연계하여 효율적으로 유방암을 예방 및 관리함과 동시에 사용자에게 대안을 제공하여 심리적 불안감을 감소시킬 수 있다.Accordingly, the breast cancer self-examination program and method according to an embodiment of the present invention can effectively prevent and manage breast cancer by linking the self-examination record with a cooperative hospital, and at the same time provide an alternative to the user to reduce psychological anxiety.
본 발명의 효과는 모바일 단말을 이용하여 사용자가 간편한 방법으로 유방암 자가 검진을 수행하고, 자가 검진의 결과로서 진료 필요 또는 정밀 진단 필요에 대한 정보를 제공하여, 조기발견을 돕고자 한다.The effect of the present invention is to help a user perform breast cancer self-examination in a convenient way using a mobile terminal, and provide information on the need for medical treatment or precise diagnosis as a result of the self-examination to help early detection.
구체적으로, 본 발명은 모바일 단말의 카메라와 연동하여, 유방암 자가 검진 순서 오디오 및 촬영 화면 상에 표시되는 유방암 검진 순서 가이드 및 증강현실로 구현된 유방 촉진 방향 가이드를 이용하여, 사용자가 유방암 자가 검진을 용이하게 수행하도록 할 수 있다.Specifically, the present invention interlocks with the camera of the mobile terminal to allow the user to perform breast cancer self-examination by using the breast cancer self-examination sequence audio and the breast cancer screening sequence guide displayed on the recording screen and the breast promotion direction guide implemented in augmented reality. can be done easily.
또한, 본 발명은 유방 이미지를 학습한 딥러닝 모델을 이용하여 사용자의 유방 이미지를 분석하여, 사용자에게 보다 정확도 높은 유방암 진료 필요성에 대한 정보를 제공할 수 있다.In addition, the present invention analyzes a user's breast image using a deep learning model that has learned the breast image, and can provide the user with information on the need for more accurate breast cancer treatment.
한편, 본 발명의 일 실시예에 따른 유방암 자가 검진 방법은 컴퓨터에 의해 실행되는 프로그램 모듈과 같은 컴퓨터에 의해 실행가능한 명령어를 포함하는 기록 매체의 형태로도 구현될 수 있다. 컴퓨터 판독 가능 매체는 컴퓨터에 의해 액세스될 수 있는 임의의 가용 매체일 수 있고, 휘발성 및 비휘발성 매체, 분리형 및 비분리형 매체를 모두 포함한다. 또한, 컴퓨터 판독가능 매체는 컴퓨터 저장 매체를 포함할 수 있다. 컴퓨터 저장 매체는 컴퓨터 판독가능 명령어, 데이터 구조, 프로그램 모듈 또는 기타 데이터와 같은 정보의 저장을 위한 임의의 방법 또는 기술로 구현된 휘발성 및 비휘발성, 분리형 및 비분리형 매체를 모두 포함한다. 본 발명의 방법 및 시스템은 특정 실시예와 관련하여 설명되었지만, 그것들의 구성 요소 또는 동작의 일부 또는 전부는 범용 하드웨어 아키텍쳐를 갖는 컴퓨터 시스템을 사용하여 구현될 수 있다.Meanwhile, the breast cancer self-examination method according to an embodiment of the present invention may also be implemented in the form of a recording medium including instructions executable by a computer, such as a program module executed by a computer. Computer-readable media can be any available media that can be accessed by a computer and includes both volatile and nonvolatile media, removable and non-removable media. Also, computer-readable media may include computer storage media. Computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Although the methods and systems of the present invention have been described with reference to specific embodiments, some or all of their components or operations may be implemented using a computer system having a general purpose hardware architecture.
이상의 설명은 본 발명의 기술적 사상을 예시적으로 설명한 것에 불과한 것으로서, 본 발명이 속하는 기술 분야에서 통상의 지식을 가진 자라면 본 발명의 본질적인 특성에서 벗어나지 않는 범위에서 다양한 수정 및 변형이 가능할 것이다. 따라서, 본 발명에 개시된 실시예는 본 발명의 기술 사상을 한정하기 위한 것이 아니라 설명하기 위한 것이고, 이러한 실시예에 의하여 본 발명의 기술 사상의 범위가 한정되는 것은 아니다. 본 발명의 보호 범위는 아래의 청구범위에 의하여 해석되어야 하며, 그와 동등한 범위 내에 있는 모든 기술적 사상은 본 발명의 권리범위에 포함되는 것으로 해석되어야 할 것이다.The above description is merely illustrative of the technical idea of the present invention, and various modifications and variations will be possible without departing from the essential characteristics of the present invention by those skilled in the art to which the present invention pertains. Therefore, the embodiments disclosed in the present invention are not intended to limit the technical spirit of the present invention, but to explain, and the scope of the technical spirit of the present invention is not limited by these embodiments. The protection scope of the present invention should be interpreted by the following claims, and all technical ideas within the scope equivalent thereto should be construed as being included in the scope of the present invention.
[부호의 설명][Explanation of code]
100: 유방암 자가 검진 서버100: breast cancer self-examination server
110: 통신 모듈110: communication module
120: 메모리120: memory
130: 프로세서130: processor
140: 데이터베이스140: database

Claims (13)

  1. 유방암 자가 검진 서버를 이용한 유방암 자가 검진 방법에 있어서,In the breast cancer self-examination method using the breast cancer self-examination server,
    사용자 단말로부터 건강 정보를 수신하는 단계;receiving health information from a user terminal;
    상기 사용자 단말로 증강현실을 이용한 유방 이미지 촬영 가이드 정보를 송신하는 단계;transmitting breast image capturing guide information using augmented reality to the user terminal;
    상기 송신된 가이드 정보에 따라 촬영된 유방 이미지를 상기 사용자 단말로부터 수신하는 단계;receiving a breast image photographed according to the transmitted guide information from the user terminal;
    상기 유방 이미지를 송신한 사용자 단말로 유방 자가 검진 기록부를 송신하는 단계;transmitting a breast self-examination record unit to the user terminal that has transmitted the breast image;
    상기 유방 자가 검진 기록부를 수신한 사용자 단말로부터 상기 유방 자가 검진 기록부에 기록된 유방 자가 검진 기록을 수신하는 단계; 및receiving a breast self-examination record recorded in the breast self-examination record unit from a user terminal that has received the breast self-examination record unit; and
    상기 사용자 단말로부터 수신한 건강 정보, 유방 이미지 및 유방암 자가 검진 기록을 기초로 제공되는 의료진 상담 필요 의견, 정밀 검진 필요 의견을 포함하는 검진 의견을 상기 사용자 단말로 송신하는 단계를 포함하는, 유방암 자가 검진 방법.Breast self-examination comprising the step of transmitting, to the user terminal, a checkup opinion including an opinion on the need for consultation with a medical staff provided based on the health information received from the user terminal, a breast image, and a record of breast cancer self-examination, and an opinion on the need for a detailed examination Way.
  2. 제1항에 있어서,According to claim 1,
    상기 건강 정보는 유방암과 연관된 문진 정보를 포함하는, 유방암 자가 검진 방법.The health information includes questionnaire information related to breast cancer, breast cancer self-examination method.
  3. 제2항에 있어서,3. The method of claim 2,
    상기 유방암과 연관된 문진 정보는 유방암 자가 검진 시기, 월경 여부, 월경일에 기초한 유방 자가 검진 시기, 유방암 가족력 여부, 출산 여부, 30세 이후 첫 출산 여부, 비만 여부, 여성호르몬 투여 여부, 유방 방사선 치료 여부 및 유방 악성 종양 여부 중 적어도 어느 하나를 포함하는, 유방암 자가 검진 방법.The breast cancer-related questionnaire information includes breast cancer self-examination time, menstruation, breast self-examination time based on menstrual date, breast cancer family history, childbirth, first childbirth after 30 years old, obesity, female hormone administration, breast radiation treatment And breast cancer self-examination method comprising at least any one of whether or not a malignant tumor.
  4. 제1항에 있어서,According to claim 1,
    상기 유방 이미지 촬영 가이드 정보는, 유방 촉진 방법을 알려주는 음성 정보 및 이미지 정보를 포함하는, 유방암 자가 검진 방법.The breast imaging guide information includes voice information and image information indicating a breast palpation method.
  5. 제4항에 있어서,5. The method of claim 4,
    상기 음성 정보는 유방 촉진 순서에 따라 출력되는 화면에 매칭되어 출력되는 유방 촉진 방법 안내 음성을 포함하는, 유방암 자가 검진 방법.The voice information includes a breast palpation method guide voice output by matching the output screen according to the breast palpation order.
  6. 제4항에 있어서,5. The method of claim 4,
    상기 이미지 정보는 상기 사용자 단말의 카메라 촬영 화면에 표시되는 유방 배치 가이드 표시 및 유방 촉진 방향 가이드 표시 중 적어도 어느 하나를 포함하는, 유방암 자가 검진 방법.The image information includes at least one of a breast placement guide display and a breast promotion direction guide display displayed on a camera photographing screen of the user terminal.
  7. 제1항에 있어서,According to claim 1,
    미리 수집된 유방 이미지 데이터를 기초로 학습된 딥러닝 모델을 통해 카메라 화면 상에 표시되는 사용자의 유방을 인식하고, 상기 인식된 유방 이미지를 기초로 상기 증강현실을 이용한 유방 이미지 촬영 가이드 정보를 제공하되,Recognizing the user's breast displayed on the camera screen through a deep learning model learned based on pre-collected breast image data, and providing breast image shooting guide information using the augmented reality based on the recognized breast image ,
    상기 미리 수집된 유방 이미지 데이터는 유방의 전체 범위, 유두, 유륜, 쇄골 및 겨드랑이 중 어느 하나로 레이블링된 유방 이미지 데이터인, 유방암 자가 검진 방법.The breast image data collected in advance is breast image data labeled with any one of the full range of the breast, nipples, areolas, clavicle, and armpit, breast cancer self-examination method.
  8. 제1항에 있어서,According to claim 1,
    상기 검진 의견이 의료진 상담 필요 또는 정밀 검진 필요 중 어느 하나인 경우에, 상기 사용자 단말의 화면에 가까운 병원 안내 정보, 병원 통화 연결부, 병원 서버로 검진 의견을 전송하기 위한 검진 의견 전송부를 포함하는 유저 인터페이스가 제공되도록 하는, 유방암 자가 검진 방법.A user interface including a checkup opinion transmission unit for transmitting the examination opinion to a hospital guide information close to the screen of the user terminal, a hospital call connection unit, and a hospital server when the examination opinion is any one of the need for consultation with a medical staff or the need for a detailed examination A breast cancer self-examination method to be provided.
  9. 제1항에 있어서,According to claim 1,
    상기 자가 검진 기록은 상기 유방 이미지 상에 표시되고, 유방암 증상 정보에 따라 다른 형상을 가지는 마크를 포함하는, 유방암 자가 검진 방법.The self-examination record is displayed on the breast image and includes a mark having a different shape according to breast cancer symptom information.
  10. 제9항에 있어서,10. The method of claim 9,
    상기 유방암 증상 정보는 통증 없는 멍울, 유방 및 겨드랑이의 덩어리, 유두의 피 섞인 분비물, 젖꼭지의 습진, 유방 피부 변화, 유두 함몰, 유방 크기 변화, 유두 위치 변화 및 유방 통증 중 적어도 어느 하나를 포함하는, 유방암 자가 검진 방법.The breast cancer symptom information includes at least one of painless lumps, lumps in the breast and armpits, bloody discharge from the nipples, eczema of the nipples, breast skin changes, nipple depression, breast size changes, nipple position changes, and breast pain. How to self-examine for breast cancer.
  11. 제1항에 있어서,According to claim 1,
    상기 사용자 단말로부터 상기 유방 이미지를 수신한 날짜 및 상기 건강 정보에 포함된 월경 정보를 기초로 예정 검진일을 계산하고, 상기 예정 검진일에 대응하여 상기 사용자 단말에 알림을 송신하는, 유방암 자가 검진 방법.A breast cancer self-examination method for calculating a scheduled check-up date based on a date the breast image is received from the user terminal and menstruation information included in the health information, and transmitting a notification to the user terminal in response to the scheduled check-up date .
  12. 유방암 자가 검진 서버에 있어서,In the breast cancer self-examination server,
    유방암 자가 검진 프로그램이 기록된 메모리; 및a memory in which a breast cancer self-examination program is recorded; and
    상기 메모리에 기록된 프로그램을 실행하는 프로세서를 포함하고,A processor for executing the program recorded in the memory;
    상기 프로세서는 상기 프로그램의 실행에 따라,The processor according to the execution of the program,
    상기 사용자 단말로 증강현실을 이용한 유방 이미지 촬영 가이드 정보를 송신하고, 상기 송신된 가이드 정보에 따라 촬영된 유방 이미지를 상기 사용자 단말로부터 수신하고, 상기 유방 이미지를 송신한 사용자 단말로 유방 자가 검진 기록부를 송신하고, 상기 유방 자가 검진 기록부를 수신한 사용자 단말로부터 상기 유방 자가 검진 기록부에 기록된 유방 자가 검진 기록을 수신하고, 상기 사용자 단말로부터 수신한 건강 정보, 유방 이미지 및 유방암 자가 검진 기록을 기초로 제공되는 의료진 상담 필요 의견, 정밀 검진 필요 의견을 포함하는 검진 의견을 상기 사용자 단말로 송신하는, 유방암 자가 검진 서버.Transmitting guide information for capturing a breast image using augmented reality to the user terminal, receiving a breast image photographed according to the transmitted guide information from the user terminal, and a breast self-examination record unit to the user terminal that has transmitted the breast image Transmitting, receiving the breast self-examination record recorded in the breast self-examination recorder from the user terminal receiving the breast self-examination record, and providing based on the health information, breast image and breast cancer self-examination record received from the user terminal A breast cancer self-examination server that transmits a checkup opinion including an opinion on the need for consultation with medical staff and an opinion on the need for a detailed examination to the user terminal.
  13. 제1항 내지 제11항 중 어느 하나의 항에 따른 방법을 구현하기 위한 컴퓨터로 판독 가능한 기록매체.A computer-readable recording medium for implementing the method according to any one of claims 1 to 11.
PCT/KR2020/016073 2019-12-12 2020-11-16 Breast cancer self-examination server and method therefor WO2021118084A2 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
KR1020190165263A KR102344659B1 (en) 2019-12-12 2019-12-12 Server and method for self-examination of breast cancer
KR10-2019-0165263 2019-12-12

Publications (2)

Publication Number Publication Date
WO2021118084A2 true WO2021118084A2 (en) 2021-06-17
WO2021118084A3 WO2021118084A3 (en) 2021-08-05

Family

ID=76329012

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/KR2020/016073 WO2021118084A2 (en) 2019-12-12 2020-11-16 Breast cancer self-examination server and method therefor

Country Status (2)

Country Link
KR (1) KR102344659B1 (en)
WO (1) WO2021118084A2 (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR102684755B1 (en) * 2021-10-15 2024-07-12 충남대학교병원 Method of providing guide for breast cancer self diagnosis and breast massaging and breast cancer self diagnosis and breast massaging device using augmented reality

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
AU2004251359B2 (en) * 2003-06-25 2009-01-22 Siemens Medical Solutions Usa, Inc. Systems and methods for automated diagnosis and decision support for breast imaging
KR20120031757A (en) * 2010-09-27 2012-04-04 엘지이노텍 주식회사 System for self-diagnosing disease
KR20140098695A (en) * 2013-01-31 2014-08-08 서울대학교산학협력단 Method for breast cancer risk assessment
KR101833147B1 (en) * 2016-11-22 2018-02-27 김수곤 Apparatus, server and method for providing of self-diagnosis result and medical information
KR101778495B1 (en) * 2017-06-28 2017-09-26 김명철 Health medical examination service system using personalized questionnaire

Also Published As

Publication number Publication date
KR20210074514A (en) 2021-06-22
KR102344659B1 (en) 2021-12-30
WO2021118084A3 (en) 2021-08-05

Similar Documents

Publication Publication Date Title
US20210166811A1 (en) Devices, Methods and Systems for Acquiring Medical Diagnostic Information and Provision of Telehealth Services
CN204428014U (en) Be convenient to the medical examination apparatus carrying out long-range inspection
US11633102B2 (en) Apparatus and method for providing improved health care
KR101051263B1 (en) Home Telehealth Home Care Service System and Method for Chronic Patients
US20100280350A1 (en) Chinese medicine tele-diagnostics and triage system
WO2017051944A1 (en) Method for increasing reading efficiency by using gaze information of user in medical image reading process and apparatus therefor
CN102231172A (en) Remote medical information acquisition system
WO2017104990A1 (en) Body fluid state diagnostic system and state diagnostic method using same
WO2013151289A1 (en) Medical image compression system and method using visually lossless compression
WO2020085576A1 (en) Method for providing health care service
WO2021118084A2 (en) Breast cancer self-examination server and method therefor
US10993625B1 (en) System, method, and apparatus for temperature asymmetry measurement of body parts
CN208422432U (en) A kind of remote diagnosis system
CN208570121U (en) Remote health self-service terminal equipment
WO2018147674A1 (en) Apparatus and method for diagnosing medical condition on basis of medical image
CN107669242A (en) A kind of intelligent health detecting system and method based on infrared thermal imaging
WO2022055037A1 (en) Tongue diagnostic kit for color correction and saliva examination, and non-face-to-face tongue diagnostic system using same
CN205597896U (en) Traditional chinese medical science diagnosis state of an illness collection system
CN207818204U (en) It is integrated with the intelligent mobile terminal of infrared scan harvester
JP2022517184A (en) Telemedicine audio video conferencing system
EP4021289A1 (en) System, method, and apparatus for temperature asymmetry measurement of body parts
WO2024038932A1 (en) Method for ecg reading service providing
WO2022211505A1 (en) Method and apparatus for providing confidence information on result of artificial intelligence model
CN214462997U (en) Remote diagnosis and treatment data acquisition system based on epidemic situation normal state prevention and control
WO2023229191A1 (en) Non-face-to-face disease diagnosis method and apparatus

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20900665

Country of ref document: EP

Kind code of ref document: A2

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20900665

Country of ref document: EP

Kind code of ref document: A2