KR20240021622A - Method and server for processing voices embedded in videos - Google Patents

Abstract

Obtaining a target video related to medical practice and including audio; Obtaining white keywords highly related to the medical practice; determining a white word corresponding to the white keyword among a plurality of wordings included in the voice; and performing voice processing to mute problem sections other than the white section corresponding to the white wording among all sections of the voice. A method, server, and device comprising a.

Description

Method and server for processing voices embedded in videos}

The technical field of the present disclosure relates to a method of processing audio included in a medical video, which includes determining a major section from the audio of the entire video section, distinguishing between a major section and a non-main section, and processing the audio in the non-main section to be muted. It is related to a technical field that provides a method of providing audio signal information included in audio-processed video to a user account using a mobile communication network between a user account and a network server.

Recently, endoscopic devices equipped with cameras have been used in various fields, and video devices that are inserted into the patient's body or film the procedure or surgery site have become miniaturized and popular, and the number of cases of video acquisition in internal and surgical surgeries is increasing. In addition, recently, with the development of big data and artificial intelligence, videos have been processed into medical information content, and based on this, quick search and analysis of key situations in videos, such as standardization research on technology and confirmation of surgical situations regarding the usability of tools, etc. It is being demanded. In addition, in the case of surgical evaluations such as medical review and evaluation, videos have been analyzed and evaluated based on review request data consisting of text and video. However, this has the limitation of making accurate confirmation difficult because the evaluation is based on limited information. . Additionally, audio signals generated during a procedure or surgical situation may be important, but unnecessary audio may be mixed in the video. Accordingly, there is a problem that videos containing unnecessary audio may contain unnecessary data volume, and problems with personal infringement may also occur. Therefore, it may be efficient to obtain information by editing the video or selectively consuming audio/video data for some sections. When a user listens to a voice in a video or checks a video, the voice included in the video can be checked so that only meaningful voices are included, so that only the more important voices and necessary parts can be checked. Therefore, there is a need to provide a method for distinguishing between main and non-main sections and providing only the voice of the main section.

Korean Patent Publication No. 10-2021-0120936 (2021.10.07) Voice interaction method, device, electronic device, readable storage medium, and computer program product

The problem to be solved in this disclosure is to process the voice included in the video, by determining and distinguishing the main section and non-main section according to the voice in the video and performing muting processing on the voice signal in the non-main section, so that meaningful voice is heard. The purpose is to provide a method and service for providing the video according to communication using a mobile communication network with the user's account.

The problems to be solved by this disclosure are not limited to the technical problems described above, and other technical problems may exist.

As a technical means for achieving the above-mentioned technical problem, the method of processing voice included in a video by a device and/or server according to the first aspect of the present disclosure is related to medical practice and involves processing a target video including voice. acquiring; Obtaining white keywords highly related to the medical practice; determining a white word corresponding to the white keyword among a plurality of wordings included in the voice; and performing voice processing to mute problem sections other than the white section corresponding to the white wording among all sections of the voice.

Additionally, the white keyword includes a medical term, and the step of performing the voice processing may mute the voice for all sections other than the white section among all sections of the voice.

Additionally, the white section may include a section corresponding to a sentence including the white wording.

Additionally, acquiring black keywords with low relevance to the medical practice; and determining a black word corresponding to the black keyword among the plurality of wordings included in the voice, wherein performing the voice processing includes processing the voice so that a black section corresponding to the black wording is muted. can be performed.

Additionally, determining the contribution to medical practice of a plurality of people included in the target video for a plurality of viewpoints; It may further include performing voice processing to mute the voice acquired from a person whose contribution to medical practice is less than a preset level.

Additionally, storing and converting the white wording corresponding to the white keyword into text; determining whether the white keyword corresponds to a preset surgery; and, if the white keyword corresponds to the preset surgery, providing an index result for the white wording converted into text.

Additionally, acquiring black keywords with low relevance to the medical practice; determining a black word corresponding to the black keyword among the plurality of wordings included in the voice; And determining a security level for the voice processing; further comprising performing the voice processing, based on the security level, analyzing the entire section other than the white section, the black section corresponding to the black wording, and the image. The audio for at least one section of the black video section determined through can be muted.

Additionally, when the black section or the black video section overlaps with the white section, audio for the overlapping section may be muted based on the security level.

A device and/or server according to a second aspect of the present disclosure includes a receiving unit that acquires a target video that is related to medical practice and includes audio; and obtaining a white keyword highly relevant to the medical practice, determining a white word corresponding to the white keyword among a plurality of wordings included in the voice, and selecting a white word other than the white wording corresponding to the white wording among the entire section of the voice. The problem section includes a processor that performs voice processing to mute; may include.

Additionally, the white keyword includes a medical term, and the processor may mute the entire voice section other than the white section among all sections of the voice.

According to the third aspect of the present disclosure, it may include a computer-readable non-transitory recording medium on which a program for implementing the first aspect is recorded.

According to an embodiment of the present disclosure, since meaningful audio areas of the surgical image are arranged in time series and provided, the user's satisfaction can be improved in that only audio with unnecessary audio signals filtered out can be provided.

In addition, efficiency can be improved because private and subjective voices in surgical videos are filtered, and concerns about infringement on doctors or nurses' personal conversations can be reduced.

Additionally, efficiency can be improved in that a voice section with high accuracy can be determined because the voice region containing medical terms is determined and only the voice from that region is provided.

In addition, efficiency can be improved in that speech in unnecessary areas can be filtered by determining the speech area containing private or subjective terms and muting the speech area in that area.

In addition, efficiency can be improved in that it is possible to provide voices with high relevance and accuracy to the surgery by distinguishing and providing the voices of only the medical staff participating in the surgery in the surgical video.

In addition, because index results are provided so that some sections according to medical terminology can be provided according to the user's selection, the user's satisfaction can be improved in that the user can check the image and audio of the area he or she wants to check.

In addition, the range of the voice provided can be adjusted by muting the voice in different ways depending on the security level of the video input from the user, so the user can be provided with a mute area and range that are adjusted according to the importance and security of the video, thereby providing satisfaction. can be improved.

The effects of the present disclosure are not limited to the effects mentioned above, and other effects not mentioned may be clearly understood by those skilled in the art from the description below.

1 is a schematic configuration diagram of a system that performs audio processing included in a video including a device and/or a server according to an embodiment.
Figure 2 is a block diagram schematically showing the configuration of a server according to an embodiment.
Figure 3 is a flowchart illustrating each step in which a device and/or server operates according to an embodiment.
FIG. 4 is a flowchart illustrating each step in which a device and/or a server performs voice processing according to an embodiment.
FIG. 5 is a diagram illustrating an example in which a device and/or a server determines a mute period, according to an embodiment.
FIG. 6 is a diagram illustrating an example in which a device and/or a server determines an overlapping section as a mute section according to a security level, according to an embodiment.
FIG. 7 is a diagram illustrating an example in which a device and/or server provides index results according to an embodiment.

Advantages and features in the present disclosure, and methods for achieving them, will become clear by referring to the embodiments described in detail below along with the accompanying drawings. However, the present disclosure is not limited to the embodiments disclosed below and may be implemented in various different forms. The present embodiments are merely provided to ensure that the disclosure is complete and to those skilled in the art. It is provided to provide complete information.

The terms used herein are for describing embodiments and are not intended to limit the disclosure. As used herein, singular forms also include plural forms, unless specifically stated otherwise in the context. As used in the specification, “comprises” and/or “comprising” does not exclude the presence or addition of one or more other elements in addition to the mentioned elements. Like reference numerals refer to like elements throughout the specification, and “and/or” includes each and every combination of one or more of the referenced elements. Although “first”, “second”, etc. are used to describe various components, these components are of course not limited by these terms. These terms are merely used to distinguish one component from another. Therefore, it goes without saying that the first component mentioned below may also be the second component within the technical spirit of the present disclosure.

Unless otherwise defined, all terms (including technical and scientific terms) used in this specification may be used with meanings commonly understood by those skilled in the art. Additionally, terms defined in commonly used dictionaries are not interpreted ideally or excessively unless clearly specifically defined.

Spatially relative terms such as “below”, “beneath”, “lower”, “above”, “upper”, etc. are used as a single term as shown in the drawing. It can be used to easily describe the correlation between a component and other components. Spatially relative terms should be understood as terms that include different directions of components during use or operation in addition to the directions shown in the drawings. For example, if a component shown in a drawing is flipped over, a component described as “below” or “beneath” another component will be placed “above” the other component. You can. Accordingly, the illustrative term “down” may include both downward and upward directions. Components can also be oriented in other directions, so spatially relative terms can be interpreted according to orientation.

Hereinafter, embodiments will be described in detail with reference to the drawings.

FIG. 1 is a diagram illustrating a system that performs audio processing included in a video including a device 100 and/or a server 190 according to an embodiment.

As shown in Figure 1, the voice processing system includes an information acquisition device 110, a device 100, an internal server 190, an external server 130, a storage medium 140, a communication device 150, and a virtual server. 160, a user terminal 170, and a network may be included.

However, those skilled in the art will understand that other general-purpose speech processing systems may include other components in addition to those shown in FIG. 1. For example, the voice processing system may further include a blockchain server (not shown) that operates in conjunction with the network. Alternatively, according to another embodiment, those skilled in the art may understand that some of the components shown in FIG. 1 may be omitted.

The voice processing device 10 including the device 100 and/or the internal server 190 according to an embodiment may obtain information related to medical procedures such as surgery from the information acquisition device 110. The information acquisition device 110 may include, but is not limited to, a photographing device, a recording device, a biological signal acquisition device, etc. Biological signals may include signals obtained from living organisms, such as body temperature signals, pulse signals, respiration signals, blood pressure signals, electromyography signals, and brain wave signals, without limitation. The imaging device, which is an example of the information acquisition device 110, may include a first imaging device (e.g., CCTV, etc.) for photographing the entire operating room situation and a second imaging device (e.g., endoscope, etc.) for intensively photographing the surgical site. may, but is not limited to this.

The device 100 according to an embodiment may acquire images (videos, still images, etc.) related to medical procedures such as surgery from the information acquisition device 110. The device 100 may perform audio processing included in the acquired image. Voice processing according to one embodiment may include naming, encoding, storage, transmission, editing, muting, metadata creation, etc. for each video, but is not limited thereto.

The voice processing device 10 according to an embodiment may transmit the medical treatment-related information obtained from the information acquisition device 110 as is or as updated information to the network. Transmission information transmitted by the voice processing device 10 to the network may be transmitted to external devices 130, 140, 150, 160, and 170 through the network. For example, the network transmits the transmission information sent by the device 100 to the network as is, such as the external server 130, the storage medium 140, the communication device 150, the virtual server 160, and the user terminal 170. Alternatively, it can be updated and transmitted. Device 100 may also receive information (eg, feedback information, update request, etc.) received from external devices 130, 140, 150, 160, and 170. The communication device 150 may refer to a device used for communication without limitation (eg, a gateway), and the communication device 150 may communicate with a device that is not directly connected to the network, such as the user terminal 180.

The device 100 and/or the server 190 according to an embodiment may include an input unit, an output processor, memory, etc., and may also include a display device (not shown), as will be described later. For example, through the display device, the user can view communication status, memory usage status, power status (e.g., battery state of charge, external power supply, etc.), thumbnails for stored videos, currently operating mode, etc. You can check, etc. Meanwhile, display devices include liquid crystal display, thin film transistor-liquid crystal display, organic light-emitting diode, flexible display, and 3D display. display), electrophoretic display, etc. Additionally, the display device may include two or more displays depending on the implementation type. Additionally, when the touchpad of the display has a layered structure and is configured as a touch screen, the display can be used as an input device in addition to an output device.

Additionally, networks may communicate with each other through wired or wireless communication. For example, a network may be implemented as a type of server and may include a Wi-Fi chip, Bluetooth chip, wireless communication chip, NFC chip, etc. Of course, the device 100 and/or the server 190 may communicate with various external devices using a Wi-Fi chip, a Bluetooth chip, a wireless communication chip, an NFC chip, etc. Wi-Fi chips and Bluetooth chips can communicate using Wi-Fi and Bluetooth methods, respectively. When using a Wi-Fi chip or a Bluetooth chip, various connection information such as SSID and session key are first transmitted and received, and various information can be transmitted and received after establishing a communication connection using this. Wireless communication chips can perform communication according to various communication standards such as IEEE, ZigBee, 3G (3rd Generation), 3GPP (3rd Generation Partnership Project), and LTE (Long Term Evolution). The NFC chip can operate in the NFC (Near Field Communication) method using the 13.56MHz band among various RF-ID frequency bands such as 135kHz, 13.56MHz, 433MHz, 860~960MHz, 2.45GHz, etc.

The input unit according to one embodiment may refer to a means through which a user inputs data to control the device 100 and/or the server 190. For example, the input unit includes a key pad, dome switch, and touch pad (contact capacitive type, pressure resistance type, infrared detection type, surface ultrasonic conduction type, integral tension measurement type, There may be a piezo effect method, etc.), a jog wheel, a jog switch, etc., but it is not limited thereto.

The output unit according to one embodiment may output an audio signal, a video signal, or a vibration signal, and the output unit may include a display device, a sound output device, and a vibration motor.

The user terminal 170 according to an embodiment may include, but is not limited to, various wired and wireless communication devices such as a smartphone, SmartPad, and tablet PC.

The voice processing device 10 according to an embodiment may update medical practice-related information obtained from the information acquisition device 110. For example, the device 100 may perform naming, encoding, storage, transmission, editing, metadata creation, etc. on the image acquired from the information acquisition device 110. As an example, the device 100 may name an image file using metadata (eg, creation time) of the acquired image. As another example, the voice processing device 10 may classify images related to medical procedures obtained from the information acquisition device 110. The voice processing device 10 can use learned AI to classify images and audio signals related to medical procedures based on various criteria such as type of surgery, operator, and location of surgery.

In one embodiment, the device 100 and the server 190 may be interconnected, and the configuration for performing voice processing with the voice processing device 10 may be performed by the server 190 and performed by the device 100. It could be. For example, device 100 may operate as a server 190. Hereinafter, it will be described later as the server 190, but in actual operation, the operating entity may be interpreted as the device 100 or the voice processing device 10, not the server 190.

Figure 2 is a block diagram schematically showing the configuration of the server 190 according to an embodiment.

Referring to FIG. 2 , the server 190 may include a receiving unit 210, a processor 220, an output unit 230, and a memory 240.

The receiving unit 210 according to an embodiment may obtain a target video that is related to medical practice and includes audio from the user account or the information acquisition device 110. In one embodiment, the target video refers to a surgery video and may include the entire section of the surgery video in which audio within the surgery video is provided.

The processor 220 according to one embodiment may acquire white keywords that are highly relevant to medical practice. Additionally, the processor 220 may determine a white wording corresponding to the white keyword among a plurality of wordings included in the voice in the target video obtained from the receiver 210. Additionally, the processor 220 can determine a white section corresponding to white wording among all sections of the voice and perform voice processing to mute problem sections other than the white section.

The output unit 230 according to one embodiment may output a voice-processed video for the target video obtained from the processor 220 to the user account. Therefore, users can monitor voice-processed videos.

The memory 240 according to an embodiment includes various information generated in the process of acquiring a voice-processed video, such as a target video acquired from the receiver 210, a white keyword acquired from the processor 220, white wording, a white section, and a problem section. Information can be stored and included.

In addition, the server 190 obtains the target video from the user account or the information acquisition device 110 at the receiver 210 and outputs the audio processed video for the target video to the user account through the output unit 230. It should be noted that it can be combined by various conventional network combinations such as Internet networks or mobile communication networks, and there are no special restrictions on this.

In addition, those skilled in the art can understand that other general-purpose configurations other than the components shown in FIG. 2 may be included in the server 180. For example, the server 190 may further include a plurality of memories (not shown) that store information about a target video that is related to a user account or a medical procedure obtained from the information acquisition device 110 and includes audio. You can. Alternatively, according to another embodiment, those skilled in the art may understand that some of the components shown in FIG. 2 may be omitted.

The server 190 according to one embodiment can be used by a user, and can be used by any type of device equipped with a touch screen panel, such as a mobile phone, smartphone, PDA (Personal Digital Assistant), PMP (Portable Multimedia Player), tablet PC, etc. It can be linked to handheld-based wireless communication devices, and is also included in devices that have a basis for installing and running applications, such as desktop PCs, tablet PCs, laptop PCs, and IPTVs, including set-top boxes. It can be linked.

The server 190 may be implemented as a terminal such as a computer that operates through a computer program to realize the functions described in this specification.

The server 190 according to an embodiment may include, but is not limited to, a system (not shown) that performs audio processing included in a video, a related server (not shown), and a related device. The server 190 according to one embodiment may support an application that performs voice processing.

Hereinafter, the description will focus on an embodiment in which the server 190 according to an embodiment independently performs voice processing included in a video. However, as described above, it may also be performed through linkage with a device. That is, the server 190 and the device 100 according to one embodiment may be integrated and implemented in terms of their functions, and the device 100 may be omitted and described, and is not limited to any one embodiment. Able to know.

FIG. 3 is a flowchart illustrating each step in which the device 100 and/or the server 190 operate according to an embodiment.

In step S310, the server 190 may acquire a target video that is related to medical practice and includes audio. In one embodiment, the target video may include a surgery video and may include all sections of the surgery video including audio. Accordingly, the server 190 may obtain a target video including audio of the entire surgical period from the user account or the information acquisition device 110.

In step S320, the server 190 may obtain a white keyword highly related to medical practice. White keywords can be determined and obtained based on an artificial intelligence-based algorithm that identifies keywords related to medical practice, and can also be preset by the administrator. Additionally, white keywords may be determined differently depending on the type and area of surgery included in the surgery video. In one embodiment, white keywords may include medical terms. Medical terms may refer to medical procedures and surgical terminology used during surgery depending on the type of surgery and area of surgery.

In step S330, the server 190 may determine a white word corresponding to the white keyword among a plurality of wordings included in the voice. For example, the white keyword may be the same word as the wording, and the white wording may correspond to a word containing a pre-acquired white keyword or the same word among a plurality of words included in the voice included in the video. Accordingly, the server 190 can determine white wording in the voice included in the video.

In step S340, the server 190 may perform voice processing to mute problem sections other than the white section corresponding to the white wording among all sections of the voice. In one embodiment, the white section may include a section according to a preset range. For example, the server 190 may obtain the time point at which white wording is included in the audio area in the video and determine the section of the preset time region (range) forward and backward from the time point at which the white wording is included as the white section. Additionally, in one embodiment, the white section may include a section corresponding to a sentence including white wording. For example, the server 190 may obtain the position containing white wording in the voice area in the video and determine the sentence area containing the white wording as the white section. Additionally, the number and position of sentences may be preset. In one embodiment, the server 190 may determine only the corresponding sentence containing white wording as a white section, and may determine the corresponding sentence containing white wording, the immediately preceding sentence, and the immediately following sentence as white according to the number of preset sentences and the preset position. It can also be decided by section. For example, if the number of preset sentences is 2 and the preset position is 'front', the corresponding sentence containing white wording and the two sentences immediately preceding it may be determined as the white section. In addition, if the preset number is 3 and the preset positions are 'front' and 'back', the corresponding sentence containing white wording, the immediately preceding sentence, and the immediately following sentence may be determined as the white section. The problem section refers to an area excluding the white section and may refer to an unnecessary area that does not correspond to a white keyword that is highly relevant to medical practice.

Therefore, the server 190 according to one embodiment may mute the audio included in the problem section excluding the white section determined from the audio of all sections in the video. Additionally, the server 190 may perform mute processing to mute the voice according to various methods. A method of performing mute processing can be explained with reference to FIG. 4.

FIG. 4 is a flowchart illustrating each step in which the device 100 and/or the server 190 perform voice processing according to an embodiment.

Referring to FIG. 4, as described above, in step S411, the server 190 may obtain a white keyword including a medical term, and in step S413, the server 190 may obtain a white wording corresponding to the white keyword including a medical term. And voice processing may be performed so that the voice for the entire section other than the white section is muted.

Additionally, referring to FIG. 4, in step S421, the server 190 may obtain a black keyword that has low relevance to medical practice. Black keywords may refer to keywords that are not related to medical practice or medical terminology. For example, all keywords other than medical terms may be determined as black keywords by a preset algorithm, or subjective and private keywords unrelated to medical practice, such as “pretty” and “handsome,” may be determined as black keywords. Additionally, in step S423, the server 190 may determine a black word corresponding to the black keyword among a plurality of wordings included in the voice. As described with respect to the white wording in step S330, the black wording may correspond to a word containing a pre-obtained black keyword or the same word among a plurality of wordings included in the video audio. Additionally, in step S425, the server 190 may perform voice processing so that the black section corresponding to the black wording is muted. As described for the white section in step S340, the black section can be obtained by obtaining a point in time when black wording is included in the audio area in the video, and determining a preset time area before and after the time point containing black wording as the black section. Additionally, the black section may include a section corresponding to a sentence including black wording. For example, the server 190 may obtain the location containing black wording in the audio area in the video and determine the sentence area containing black wording as the black section. Accordingly, the server 190 according to one embodiment may mute the voice included in the black section determined from the voice of the entire section in the video.

Referring to FIG. 4, in step S431, the server 190 may determine the contribution to medical practice of a plurality of people included in the target video for a plurality of viewpoints. For example, contribution to medical practice may be determined based on the locations of multiple people. For example, during surgery, there may be a preset position of the surgeon, a preset position of a sub-doctor, a preset position of an anesthetist, a preset position of a nurse, etc., and the contribution to medical practice may be determined depending on the positions of multiple people. there is. At this time, the server 190 may determine the medical practice contribution of the person located at the preset position of the operating doctor as the highest ranking and determine the medical practice contribution of the person located at the sub-doctor's preset position as the second highest ranking. You can. Additionally, the server 190 may determine the contribution to medical practice of a person located at the anesthetist's preset position as high as the third priority, and may determine the contribution to medical practice of a person located at the nurse's preset position as high as the fourth priority. Additionally, preset medical staff (eg, surgeon, sub-doctor, anesthetist, nurse, etc.) and preset locations can be flexibly determined and changed by the user or administrator. In one embodiment, the server 190 may provide a frame in which all medical staff appear in the video to the user account, and the contribution of each medical staff member to medical practice may be determined according to the user's selection input for the contribution. For example, the user may select each of a plurality of people and enter a ranking according to each medical staff member's contribution to medical practice, thereby determining the ranking according to each medical staff member's contribution to medical practice. Additionally, in step S433, the server 190 may perform voice processing to mute the voice acquired from a person whose contribution to medical practice is less than a preset level. For example, the server 190 may determine the rank corresponding to the preset level to be rank 3. Accordingly, the server 190 may mute the voice obtained from the nurse whose contribution to medical practice is ranked 4th. Additionally, the ranking corresponding to the preset level may be flexibly determined or changed by the user or administrator.

The server 190 according to one embodiment may determine the security level for voice processing. Additionally, based on the security level, the server 190 may mute the voice for at least one section among the entire section other than the white section, the black section corresponding to the black wording, and the black video section determined through video analysis. The black video section may be different from the black section. Through image analysis based on artificial intelligence-based algorithms, surgical tools and situations can be identified, and video sections corresponding to non-main sections other than the main section within the target video can be determined as black video sections. The security level may be predetermined by the user or administrator. For example, the server 190 may obtain information about the security level indicating the security level of the corresponding surgery video from the user account. The security level can be determined as 1 (high), 2 (medium), and 3 (low), and the administrator or user can enter the corresponding security level for each target video in advance. In one embodiment, when the security level of the acquired target video is 1 (high), the server 190 may determine that the surgery video is a surgery video of very high importance and there is a high need to improve stability, so You can mute the voice for the entire section. In addition, when the security level of the acquired target video is 2 (medium), the server 190 can determine that the surgery video has a certain level of importance even though the importance of the surgery video is lower than security level 1 (high), so the black You can mute the audio for sections and black video sections. Additionally, if the security level of the acquired target video is 3 (low), the server 190 may determine that it is okay to remove only the minimum amount of audio, and thus may mute only the audio for the black section. Therefore, since the server 190 determines the mute section 510 in different ways depending on the security level, the user can be provided with a voice section with high stability.

FIG. 5 is a diagram illustrating an example in which the device 100 and/or the server 190 determines a mute period 510 according to an embodiment.

Referring to FIG. 5, (a) of FIG. 5 is a diagram showing a mute section 510 determined according to the white wordings 513, 523, and 533 and the white sections 511, 523, and 533, and (in FIG. 5) b) is a diagram showing the mute section 510 determined according to the black wordings 515, 525, and 535 and the black sections 512, 522, and 532. As shown in (a) of Figure 5, in one embodiment, the server 190 includes white wordings 513, 523, and 533 corresponding to the white keywords obtained as described above in the audio area of the entire target video section. The point can be obtained and the white section (511, 521, 531) corresponding to the obtained point can be determined. Additionally, the server 190 may determine a section other than the white sections 511, 521, and 531 in the audio area of the entire target video section as the mute section 510. Accordingly, the server 190 may perform voice processing so that the voice in the determined mute section 510 is muted. Additionally, as shown in (b) of FIG. 5, in one embodiment, the mute section 510 may be determined using a method different from the method described in (a) of FIG. 5. For example, the server 190 may obtain a point containing black wordings 515, 525, 535 corresponding to the black keyword obtained as described above in the audio area of the entire target video section and correspond to the obtained point. The black sections 512, 522, and 532 can be determined. Additionally, the server 190 may determine the black sections 512, 522, and 532 in the audio area of the entire target video section as the mute section 510. Accordingly, the server 190 may perform voice processing so that the voice in the determined mute section 510 is muted.

In addition, when the black section 512, 525, 535 or the black video section and the white section 511, 521, 531 overlap, the server 190 according to one embodiment provides audio for the overlapping section based on the security level. can be muted. The server 190 according to one embodiment may mute the audio for the overlapping section when the black section 512, 522, and 532 or the black video section and the white section 511, 521, and 531 overlap. This can be explained with reference to FIG. 6.

FIG. 6 is a diagram illustrating an example in which the device 100 and/or the server 190 determines an overlapping section as a mute section according to a security level.

Referring to FIG. 6, in one embodiment, there may be a black section 512, 522 or a section where the black video section 611, 621 and the white section 511, 521, 531 overlap in the entire video audio section. . At this time, the black section (512, 522) or the black image section (611, 621) and the white section (511, 521, 531) overlap with the black section (512, 522) and the black image section (611, 621). Since the section corresponding to is included, it can be determined that there is a high probability that it is an unnecessary voice section, so the server 190 can determine the overlapping section as the mute section 510.

In one embodiment, the security level of the target video for muting the voice for the overlapping section may be determined as 1 (high), 2 (medium), 3 (low middle), 4 (low), and 5 (lowest). Administrators or users can pre-enter the security level corresponding to each target video. The server 190 may determine the mute section 510 differently depending on the security level according to the importance of the surgery video. FIG. 6 is a diagram omitting the fact that the entire section other than the white sections 511, 521, and 531 is the mute section 510 in order to show the overlapping sections between each section. Referring to FIG. 6, when the security level of the acquired target video is 1 (above), the server 190 can determine that the surgery video is a very important surgery video, so the white sections 511, 521, and 531 Any section where the black sections 512, 525, 535 or the black image sections 611, 621 overlap can be determined as the mute section 510. Therefore, all sections where the white section (511, 521, 531) overlaps with the black section (512, 525, 535) or the black video section (611,621) and all sections other than the white section (511, 521, 531) are muted sections (510). ) can be determined. In addition, since the black sections (512, 525, 535) correspond to unnecessary sections according to the voice, there is a high probability that they are more important sections than the video-based black video sections (611, 621) in determining the mute section (510). When the security level of the acquired target video is 2 (medium), the server 190 selects only the section where the white section (511, 521, 531) and the black section (512, 525, 535) overlap as the mute section (510). You can decide. At this time, the section where the white sections 511, 521, and 531 overlap with the black image sections 611 and 621 may be non-muted. Accordingly, the entire section other than the overlapping section between the white section 511, 521, and 531 and the black section 512, 525, and 535 and the white section 511, 521, and 531 may be determined as the mute section 510. In addition, when the security level of the acquired target video is 3 (low to medium), the server 190 determines only the section where the white sections 511, 521, 531 and the black video sections 611, 621 overlap as the mute section 510. You can. At this time, the section where the white sections 511, 521, and 531 overlap with the black sections 512, 525, and 535 may be non-muted. Accordingly, the entire section other than the section where the white section (511, 521, 531) and the black image section (611, 621) overlap and the white section (511, 521, 531) may be determined as the mute section (510). In addition, there may be a section where all three sections overlap in the entire target video section, and the server 190 may select the white sections 511, 521, and 531 and the black section when the security level of the acquired target video is 4 (low). The section where (512, 525, 535) and the black image section (611, 621) overlap can be determined as the mute section (510). Therefore, the white section (511, 521, 531), the black section (512, 525, 535), and the black video section (611,621) all overlap, and the entire section other than the white section (511, 521, 531) is the mute section (510). ) can be determined. In addition, when the security level of the acquired target video is 5 (lowest), the server 190 overlaps the white section (511, 521, 531) and the black section (512, 525, 535) or the black video section (611, 621). The entire section other than the white sections 511, 521, and 531 can be ignored and the entire section other than the white sections 511, 521, and 531 determined as the muted section 510, so that all voices in the white sections 511, 521, and 531 are unmuted. Accordingly, if there is an overlapping section of each section, the server 190 determines the section corresponding to the overlapping section as the mute section 510 according to the security level, thereby providing a voice section with higher accuracy.

The server 190 according to one embodiment may provide index results for white wordings 513, 523, and 533. This can be explained with reference to FIG. 7 .

FIG. 7 is a diagram illustrating an example in which the device 100 and/or the server 190 provide index results according to an embodiment.

Referring to FIG. 7, the server 190 may provide a screen that displays a plurality of white wordings in the white wording information tables 731, 732, 733, 734, 735, 736, 737, and 738 of the target video, The location of the white section areas (711, 712, 713, 714, 715, 716, 717, 718) corresponding to the white information table (731, 732, 733, 734, 735, 736, 737, 738) is provided to the user. It is possible to make intuitive confirmation of . For example, the user can select and input a desired white wording among a plurality of white wordings and enter the white section area (711, 712, 713, 714, 715, 716, 717, 718) containing the corresponding white wording. By providing the location of the area, users can immediately check video and audio in that area. Therefore, because the server 190 provides the index result, it can immediately provide the location of the white section area (711, 712, 713, 714, 715, 716, 717, 718) corresponding to the white wording for the user to check. This can improve user convenience. In addition, the white wording displayed on the white information tables 731, 732, 733, 734, 735, 736, 737, and 738 may be determined according to the type of surgery and a plurality of white words included in the target video voice, and may be determined by the administrator or user. It can also be set by .

According to one embodiment, the server 190 may store and convert white wording corresponding to the white keyword into text. The server 190 may convert the white wording into text and store the obtained text in the memory 240, etc.

According to one embodiment, the server 190 may determine whether the white keyword corresponds to a preset surgery. Additionally, if the white keyword corresponds to a preset surgery, the server 190 may provide index results for the textual white wording. For example, the server 190 may determine the stage to which the white keyword corresponds (e.g., anesthesia stage, laparotomy stage, surgery stage, etc.) and obtain index information for the corresponding stage. The server 190 may provide an updated target video by merging the index information into the target video. The process of updating the target video by adding index information can proceed automatically, and a user who has acquired the updated target video can easily access the playback time corresponding to the white keyword using the index information.

According to one embodiment, since the meaningful audio areas of the surgical image are provided by arranging them in time series, the user's satisfaction can be improved in that the user can only hear the voice with filtered out unnecessary audio signals, and the personal and subjective sound areas in the surgical image are provided. Efficiency can be improved because the voice of the content is filtered, and efficiency can be improved in that concerns about infringement on a doctor or nurse's personal conversation can be lowered. In addition, efficiency can be improved in that it is possible to determine a voice section with high accuracy by determining the voice area containing medical terms and providing only the voice of that region, and by determining the voice region containing private and subjective terms, the corresponding region Since the voice area is muted, efficiency can be improved in that voice in unnecessary areas can be filtered. In addition, efficiency can be improved in that it is possible to provide voices with high relevance and accuracy to the surgery by distinguishing and providing the voices of only the medical staff participating in the surgery in the surgery video, and some sections according to medical terminology can be selected by the user. Because index results are provided so that they can be provided according to selection, user satisfaction can be improved in that users can check video and audio in the area they want to check. In addition, the range of the voice provided can be adjusted by muting the voice in different ways depending on the security level of the video input from the user, so the user can be provided with a mute area and range that are adjusted according to the importance and security of the video, thereby providing satisfaction. can be improved.

Various embodiments of the present disclosure are implemented as software including one or more instructions stored in a storage medium (e.g., memory) that can be read by a machine (e.g., a display device or computer). It can be. For example, the device's processor (eg, processor 220) may call at least one instruction among one or more instructions stored from a storage medium and execute it. This allows the device to be operated to perform at least one function according to the at least one instruction called. The one or more instructions may include code generated by a compiler or code that can be executed by an interpreter. A storage medium that can be read by a device may be provided in the form of a non-transitory storage medium. Here, 'non-transitory' only means that the storage medium is a tangible device and does not contain signals (e.g. electromagnetic waves). This term refers to cases where data is stored semi-permanently in the storage medium. There is no distinction between temporary storage cases.

According to one embodiment, methods according to various embodiments disclosed in the present disclosure may be included and provided in a computer program product. Computer program products are commodities and can be traded between sellers and buyers. The computer program product may be distributed in the form of a machine-readable storage medium (e.g. compact disc read only memory (CD-ROM)) or via an application store (e.g. Play StoreTM) or on two user devices (e.g. It can be distributed (e.g. downloaded or uploaded) directly between smartphones) or online. In the case of online distribution, at least a portion of the computer program product may be at least temporarily stored or temporarily created in a machine-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or a relay server.

Although the present invention has been described with reference to the illustrative drawings, it is not limited to the disclosed embodiments and drawings, and those skilled in the art in the technical field related to the present embodiments may make modifications without departing from the essential characteristics of the above-mentioned description. You will be able to understand that it can be implemented in a certain form. Therefore, the disclosed methods should be considered from an explanatory rather than a restrictive perspective. Even if the operational effects according to the configuration of the present invention are not explicitly described and explained in the description of the embodiment, the effects that can be predicted by the configuration may also be recognized. The scope of the present invention is indicated in the claims rather than the foregoing description, and all differences within the equivalent scope should be construed as being included in the present invention.

100: device 190: server
210: receiving unit 220: processor
230: output unit 240: memory
511, 521, 531: White section 510: Mute section
513, 523, 533: White Wording
512, 522, 532: Black section 515, 525, 535: Black wording
611, 621: Black video section
711, 712, 713, 714, 715, 716, 717, 718: White section area
731, 732, 733, 734, 735, 736, 737, 738: White Information Table

Claims (11)

In the method of processing audio included in a video,
Obtaining a target video related to medical practice and including audio;
Obtaining white keywords highly related to the medical practice;
determining a white word corresponding to the white keyword among a plurality of wordings included in the voice; and
A method comprising: performing voice processing to mute problem sections other than the white section corresponding to the white wording among all sections of the voice. According to claim 1,
The white keywords include medical terms,
The step of performing the voice processing is
A method of muting the voice for all sections other than the white section among all sections of the voice. According to claim 1,
The white section includes a section corresponding to a sentence including the white wording. According to claim 1,
Obtaining black keywords with low relevance to the medical practice; and
Further comprising: determining a black word corresponding to the black keyword among the plurality of wordings included in the voice,
The step of performing the voice processing is
A method of performing voice processing so that the black section corresponding to the black wording is muted. According to claim 1,
determining the contribution of a plurality of people included in the target video to medical practice for a plurality of viewpoints; and
The method further comprising performing voice processing to mute the voice obtained from a person whose contribution to medical practice is less than a preset level. According to claim 1,
storing and converting the white wording corresponding to the white keyword into text;
determining whether the white keyword corresponds to a preset surgery; and
If the white keyword corresponds to the preset surgery, providing an index result for the textualized white wording. The method further comprising: According to claim 1,
Obtaining black keywords with low relevance to the medical practice;
determining a black word corresponding to the black keyword among the plurality of wordings included in the voice; and
Further comprising: determining a security level for the voice processing,
The step of performing the voice processing is
A method of muting the voice for at least one section among all sections other than the white section, a black section corresponding to the black wording, and a black video section determined through video analysis, based on the security level. According to claim 7,
When the black section or the black video section and the white section overlap, a method of muting the voice for the overlapping section based on the security level. In the server that performs audio processing included in the video,
a receiving unit that acquires a target video that is related to medical practice and includes audio; and
Obtaining a white keyword highly related to the medical practice, determining a white wording corresponding to the white keyword among a plurality of wordings included in the voice, and determining problems other than the white section corresponding to the white wording among the entire section of the voice. The section includes a processor that performs voice processing to mute; Server, including. According to clause 9,
The white keywords include medical terms,
The processor is
A server that mutes the voice for all sections other than the white section among all sections of the voice. A non-transitory computer-readable recording medium on which a program for implementing the method of any one of claims 1 to 8 is recorded.

Images