KR100963850B1 - Capsule endoscope system, and method of managing image data thereof - Google Patents

Abstract

PURPOSE: A capsule endoscope system and an image processing method thereof are provided to facilitate the diagnosis of a doctor by displaying time-series image frames corresponding to image data received from a capsule endoscope. CONSTITUTION: A capsule endoscope(10) moves along the digestive organ of a human body. A capsule endoscope generates and transmits the image signal inside the digestive organ of a human body. A receiver(20) receives an image signal from a sensor. A workstation(30) comprises a receiving part, a controlling part, and a display part. The receiving part receives an image signal from the receiver. The controlling part processes the image signal received from the receiving part. The display part displays the processing result of the controlling part in a screen.

Description

Capsule endoscope system, and method of managing image data

The present invention relates to a technique for processing an image taken by a capsule endoscope, and in particular, to generate a plurality of time series image frames from the image taken by the human body capsule endoscope and to display and change the generated time series image frame in various ways A capsule endoscope system for processing and image data processing method thereof.

Endoscopy is an implantable medical device originally designed to observe organs that cannot be seen directly without surgery or autopsy.

However, endoscopy is due to the pain and discomfort associated with the test, many patients have to avoid the endoscopy and receive medication instead. In particular, endoscopes have been used for many diagnosis and examinations, but because of the size and stiffness of the catheter, it has caused various inconveniences to patients and medical staff.

Accordingly, in recent years, capsule type endoscopes have been developed to supplement various disadvantages of endoscopes, and have been used to diagnose various diseases in the medical field.

Capsule type endoscope, or capsule endoscope, is a capsule sized endoscope that can be swallowed through the oral cavity of the human body and moved according to the peristalsis of the digestive system, which is one of the body cavities of the human body, before the image is taken. Transmitted video information to an external device through wireless communication.

These capsule endoscopes do not require anesthesia, there is no nausea, and it is possible to take pictures of the small intestine that could not be taken with the conventional general endoscope, and thus has the advantage of enabling a more accurate medical diagnosis.

The capsule endoscope is very small and takes tens of thousands of subjects while moving according to the peristalsis of the digestive system in the human body for 8 to 10 hours. The endoscope system including the capsule endoscope has a basic configuration of a workstation including a receiver for wirelessly receiving an image captured by the capsule endoscope and a monitor for displaying an image received through the receiver.

The image received through the receiver, that is, the image taken by the capsule endoscope is displayed on the monitor, and the doctor diagnoses the patient by checking the image shown on the monitor. In particular, the workstation includes software for diagnosing a patient, and accordingly, the doctor uses the software to read an image and diagnose the condition and suspicious findings.

The diagnosis result of the patient is stored in the workstation, and the workstation has a function of displaying the stored diagnosis result in an annotation form.

However, most of the images need to check thousands of tens of thousands of images, and many of them include images of unnecessary parts of the diagnosis or images that do not help the examination. Nevertheless, it was necessary to keep watching all the images displayed.

Accordingly, in the related art, a doctor has to diagnose a patient while checking all images displayed on a monitor, which wastes a lot of time and labor.

Of course, in the related art, some of the displayed images are collected and displayed separately in the form of thumbnails while the received real-time image is displayed over time. By further displaying in thumbnail form, only the areas suspected of the disease were selected and viewed at a glance.

However, in the prior art, only a plurality of still images themselves are simultaneously shown in a thumbnail form on a separate display area of the monitor screen, and diagnostic information on still images displayed on the separate display area is not supported at all. Effective diagnosis was not easy when diagnosing.

In the prior art, only the still image captured in the separate display area was diagnosed at the time of rediagnosis, and in order to check the image of the suspected part, it was necessary to vaguely check the image from the specific position again. As a result, the time required for diagnosis also increased.

Due to the above reasons, a form of displaying a simplified image more efficiently is required in recent years in order to effectively process the diagnosis and the result after diagnosis.

SUMMARY OF THE INVENTION An object of the present invention has been made in view of the above-mentioned points, and it is possible to continuously display and display a plurality of time-series sequenced image frames corresponding to image data received from an endoscope, or to display some of the image frames. It is to provide a capsule endoscope system for expanding or reducing the display and image data processing method thereof.

Another object of the present invention is to classify a number of image frames corresponding to the image data received from the capsule endoscope into a plurality of temporal sections, and to display the continuous representation of the representative video frames represented in each temporal section successively arranged in time series The present invention provides a capsule endoscope system and an image data processing method thereof.

Still another object of the present invention is to provide a capsule endoscope system and an image thereof in which a plurality of time series image frames or a plurality of section representative image frames are continuously arranged in a map structure in a vertical and horizontal order, and at the same time, the size of the displayed image frame is adjusted. To provide a data processing method.

Another object of the present invention is to selectively display any one of a full time series image frame and a plurality of time series image frames classified according to a specific criterion (an interval representative image frame, an annotated image frame, or an undeleted image frame). The present invention provides a capsule endoscope system and a method for processing image data.

Still another object of the present invention is to classify a number of image frames corresponding to the image data received from the capsule endoscope into a plurality of temporal sections, and to display a sequence of representative video frames represented in each temporal section continuously arranged in time series In addition, the present invention provides a capsule endoscope system for continuously reproducing a plurality of video frames belonging to a temporal section automatically according to a selection of a section representative video frame and a method of processing the image data.

Still another object of the present invention is to provide a capsule endoscope system and image data processing for displaying a plurality of time series video frames or a plurality of representative video frames continuously arranged in a map structure in a columnar order and displaying whether annotations are added together. To provide a way.

It is still another object of the present invention to provide a capsule for easily eliminating unnecessary video frames by selecting and deleting regions in a plurality of time series video frames or a multi-section representative video frame, and also displaying a symptom for an important video frame through marking after selecting a region. The present invention provides an endoscope system and a method of processing image data thereof.

A feature of the capsule endoscope system according to the present invention for achieving the above object is to move along the digestive organs of the human body to generate an image signal by photographing the interior of the digestive organs, and to continuously transmit the generated video signal Capsule endoscope; A receiver for receiving and storing the image signal from a sensor that detects the image signal in the human body; And a workstation including a receiver configured to receive the stored video signal from the receiver, a controller configured to process the video signal received by the receiver, and a display configured to display an image according to the processing result of the controller on a screen. The controller generates a plurality of video frames from the video signal received by the receiver; Displaying the image on the screen in which the plurality of generated image frames are continuously arranged; Setting an enlargement or reduction ratio of the plurality of image frames according to an enlargement or reduction command of the displayed image; Determining the number of multiple image frames to be displayed on the screen according to the set ratio; Enlarge or reduce the size of the determined number of image frames; The enlarged or reduced image frame is arranged to be continuously displayed on the screen.

A feature of the image data processing method according to the present invention for achieving the above object is a method for processing the image data received from the capsule endoscope by the workstation, generating a plurality of image frames from the image signal received by the capsule endoscope step; Sequentially arranging the generated plurality of image frames and displaying them on a screen; Setting an enlargement or reduction ratio of the displayed multiple image frames according to a command for adjusting the size of the displayed multiple image frames; Determining the number of multiple image frames to be displayed on the screen according to the set ratio; Enlarging or reducing the size of the determined number of image frames; And rearranging and displaying the enlarged or reduced image frame on the screen continuously.

According to the present invention, an efficient simplification of the image data is realized by continuously arranging and displaying a plurality of time series image frames corresponding to the image data received from the capsule endoscope according to the time series. In addition, after enlarging or reducing a large number of time series image frames corresponding to the image data received from the capsule endoscope, only a plurality of image frames determined according to the enlargement or reduction ratio are continuously arranged and displayed according to the time series. Efficient Simplification

In addition, a number of video frames corresponding to the image data received from the capsule endoscope is classified into a plurality of temporal sections, and the section representative video frames represented in each temporal section are arranged and displayed in sequence according to the time series, and also the section representative video frames According to the selection of the auto-play a plurality of video frames belonging to his temporal section continuously, it is possible to simply check all the video frames through the representative video frame. As a result, efficient simplification of the image data is realized.

In addition, a plurality of time series image frames or a plurality of section representative image frames are continuously arranged and arranged in a map structure in a vertical and horizontal order, and at the same time, the size of the displayed image frame is adjusted to facilitate the diagnosis of a doctor.

In addition, a transition control is provided for selectively displaying any one of the entire time series video frames and a plurality of time series video frames classified according to a specific criterion (an interval representative video frame, an annotated video frame, and an undeleted video frame). Make your images easier to see when your doctor makes a diagnosis.

In addition, a plurality of time series image frames or a plurality of representative image frames are continuously arranged and displayed in a columnar ordered map structure, and an indication of whether annotations are added is also displayed to facilitate diagnosis of a doctor's condition and suspicion.

In addition, it is possible to easily remove unnecessary video frames by selecting and deleting regions in multiple time series image frames or multiple region representative image frames, and to mark the symptoms of important image frames through marking after selection of regions. Diagnosis is simple and easy.

As a result, in the present invention, the display process for the image data can be simplified, and detailed information for diagnosis can be confirmed in more detail. It also eliminates the need to vaguely reconfirm the image from a specific location during diagnosis. Therefore, it is easier to diagnose the doctor's condition and suspicious area, and the doctor can easily make accurate diagnosis from the image data received from the capsule endoscope.

1 is a diagram showing a schematic configuration of a capsule endoscope system according to the present invention.
Figure 2 is a block diagram showing the internal configuration of the workstation in the capsule endoscope system according to the present invention.
3 to 4 are diagrams showing display examples of time series image frames in a workstation of the present invention.
5 to 7 are diagrams showing enlarged display examples of time series image frames in a workstation of the present invention.
8 is a diagram illustrating an example of selecting a part of the entire time series image frame in the workstation of the present invention and setting a comment on the selected portion thereof.
9 is a diagram illustrating an example of a mini map for a selection area according to an embodiment of the present invention.
10 is a flowchart showing an image data processing procedure according to the present invention.
FIG. 11 is a flowchart illustrating a magnification or reduction processing procedure of a time series image frame according to the present invention. FIG.
FIG. 12 is a flowchart showing a processing procedure for displaying image data in an interval representative image frame in the present invention. FIG.
FIG. 13 is a diagram showing an example of further displaying a mini map area for a display area in the present invention; FIG.

Other objects, features and advantages of the present invention will become apparent from the detailed description of the embodiments with reference to the accompanying drawings.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, a configuration and an operation of an embodiment of the present invention will be described with reference to the accompanying drawings, and the configuration and operation of the present invention shown in and described by the drawings will be described as at least one embodiment, The technical idea of the present invention and its essential structure and action are not limited.

Hereinafter, various embodiments of a capsule endoscope system and an image data processing method thereof according to the present invention will be described in detail with reference to the accompanying drawings.

1 is a diagram showing a schematic configuration of a capsule endoscope system according to the present invention.

Referring to FIG. 1, the system of the present invention is composed of a capsule endoscope 10, a receiver 20, and a workstation 30.

Capsule endoscope 10 is in the form of a capsule, approximately 11mm in diameter, 24mm in length to take an image of the digestive tract 41 in the human body 40. Thus, when the patient swallows the capsule endoscope 10, the capsule endoscope 10 moves over the digestive tract 41 over the esophagus to the anus over approximately 10 hours. During the movement, the camera provided in the capsule endoscope 10 photographs the inside of the digestive organ 41. The camera provided in the capsule endoscope 10 has an ultra precision lens having a viewing angle of 150 °.

As such, the capsule endoscope 10 moves along the digestive tract 41 of the human body 40 and photographs the inside of the digestive tract 41 to generate an image signal. Then, the generated video signal is continuously transmitted using the human body conduction method. That is, the human body 40 is transmitted as the conductor to the outside of the human body 40 continuously.

The human body conduction method described above converts the image information captured by the capsule endoscope 10 into the digital signal of 0 and 1 in the digestive organ 41, and converts the converted digital signal into an electrical signal having a polarity of positive and negative poles. Finely flows to the human body (40). The electrical signal flowing through the human body 40 corresponds to an image signal and is input to the receiver 20 outside the human body, and then restored to the image.

For example, the capsule endoscope 10 transmits the captured image to the receiver 20 in real time whenever three images are captured per second. The video transmitted at this time is a video stream.

The system according to the present invention has a sensor attached to the human body 40 so that the video signal can be input to the receiver 20. The sensor is electrically connected to the receiver 20, so that the sensor detects an image signal in the human body 40 and transmits the detected image signal to the receiver 20.

The receiver 20 receives and stores the video signal from a sensor that detects the video signal in the human body 40. The receiver 20 receives and stores an image signal from the capsule endoscope 10 until the photographing is completed in the capsule endoscope 10. The stored video signal is transmitted to the workstation 30.

In order to transmit data between the receiver 20 and the workstation 30, the receiver 20 and the workstation 30 are connected by a wired communication method such as a USB terminal or a short distance wireless communication method such as Bluetooth.

The workstation 30 may reproduce the video signal received from the receiver 20 in units of frames.

In the present invention, a plurality of time series video frames are sequentially arranged and displayed so that a plurality of time series video frames consecutive in time can be simplified and reproduced on a screen. In particular, in the continuous arrangement, a map structure arranged in the order of the vertical columns is used.

Alternatively, in the present invention, the entire time series video frame is divided into several temporal sections, and a section representative video frame representing each temporal section is selected. The selected multiple-section representative video frame is continuously arranged and displayed. Here, in the continuous arrangement, the section representative image frame may be displayed in a map structure arranged in the order of vertical columns. Here, the section representative video frame may be for automatically playing back a plurality of video frames belonging to the temporal section.

In the example of displaying the interval representative video frame in a map structure, the video frames belonging to the temporal interval are associated with the selected interval representative video frame. Here, the temporal section for selecting the section representative video frame may be randomly included to include the same number of video frames, or may be divided based on the video frame to which an annotation corresponding to a comment is added. .

Figure 2 is a block diagram showing the internal configuration of the workstation in the capsule endoscope system according to the present invention.

Referring to FIG. 2, the workstation 30 includes a receiver 31, a storage 32, a controller 33, a display 34, and a user input 35.

The user input unit 35 may be a keyboard or a mouse as a kind of user interface. In the present invention, the control unit 33 adds the information input through the user input unit 35 to the corresponding video frame as an annotation, and the image frame to which the annotation is added may be stored in the storage unit 32.

The receiver 31 receives an image signal stored therein from the receiver 20.

The storage unit 32 stores image data. The image data may be time series data in which the image signal received by the receiver 31 is defined in units of image frames, or annotation for each image frame input through the user input unit 35 to the image signal received by the receiver 31. ) May be added to time series data. Hereinafter, the control unit 33 reads the image data stored in the storage unit 32 to generate a plurality of time series image frames, and accordingly, the storage unit 32 stores the entire time series image frames generated by the control unit 33. Explain.

In addition, the storage unit 32 may store a section representative image frame representing each temporal section among the entire time series image frames in a separate storage area. In addition, it is preferable to store the remaining video frames of the corresponding temporal section in time series in association with the representative video frame of each section.

The controller 33 controls the receiver 31, the storage 32, the display unit 34, and the user input unit 35 provided in the workstation 30. That is, the controller 33 performs the control for the receiver 31 to receive the video signal from the receiver 20. In addition, the controller 33 performs control for storing the received video signal or the generated video frame time series and displaying the time series. In particular, the controller 33 performs display control such as arranging and displaying a time series image frame according to a screen through display control of the display unit 34, and displaying the time series image frame by enlarging or reducing the time series image frame. The operation of the control unit 33 will be described in detail below.

The controller 33 generates a plurality of time series image frames by reading image data stored in the storage unit 32. In some cases, a multi-section representative video frame representing a multi-temporal section is generated from the multi-time series video frame.

As a separate example, in the present invention, the control unit 33 further generates a thumbnail image that can represent each temporal section using a plurality of section representative image frames, and separately stores the sample image in the storage unit 32. You can also save.

The display unit 34 arranges and displays a plurality of time series image frames generated by the controller 33 in succession in time. In the present invention, the screen structure is displayed in the map structure in the column order.

3 to 7 illustrate various examples of displaying a time series image frame in a map structure under the control of the controller 33, and are examples of a matrix structure of N * M (Column * Row).

The controller 33 controls the display unit 34 to display the entire time series image frame on the display unit 34 or display only the section representative image frame among the entire time series image frames according to a user command through the user input unit 35. That is, the display unit 34 displays one of the entire time series video frame and the section representative video frame in a vertically arranged map structure according to the switching control of the controller 33.

Hereinafter, a screen area for displaying the entire time series image frame is defined as the display area 100. Of course, the display area 100 should also be considered to display only the section representative image frame instead of the entire time series image frame according to the switching control of the controller 33.

Accordingly, the display unit 34 continuously displays and displays a plurality of time series image frames in the display area 100 in time, which may be all time series image frames or interval representative image frames.

Meanwhile, the storage unit 32 may store the image frames of the corresponding temporal section in time series in association with the section representative image frames. Thus, the controller 33 may automatically reproduce a plurality of video frames belonging to the temporal section at a predetermined time interval through a specific section representative video frame. For example, in a state where a plurality of section representative image frames are displayed in a map structure arranged vertically and horizontally in the display area 100, when one section representative image frame is selected, the control unit 33 selects the selected section representative image frame. Automatically play multiple linked video frames continuously.

In addition, in the automatic playback according to the selection of the interval representative video frame, the controller 33 displays at least one time series video frame that is automatically played vertically or horizontally. In addition, the control unit 33 further provides a first icon for forward forward playback in time during automatic playback, a second icon for backward forward playback in time, and a third icon for stopping automatic playback through the display unit 34. can do.

3 to 4 are diagrams showing display examples of time series image frames in a workstation of the present invention.

As shown in FIGS. 3 and 4, the present invention displays the entire time series image frame or the interval representative image frame in a map structure arranged in a vertical column.

If the total time series video frames shown in Figs. 3 and 4 are N * M (Column * Row), the control unit 33 arranges the M video frames which are the fastest in time in the first column, and then in the horizontally arranged column. Shifting (column), i.e., shifting to the second column, to arrange the next M video frames downwards, and then shifting one column to the horizontal column, that is, shifting to the third column, to rearrange the next M video frames. In this manner, the M-frames are arranged downward in the column and displayed in the display area 100 while shifting the columns by one column.

As another example, when the entire time series image frame is N * M (Column * Row), the controller 33 arranges the N image frames which are the fastest in time in the first row, and then vertically arranges them. Shift the row, i.e. shift to the second row, to align the next N video frames to the right, and then shift the row back to the vertical row, ie shift to the third row, to align the next N video frames to the right. do. In this manner, the image frames may be arranged to be right-aligned in each row while shifting one row in a row, and may be controlled to be displayed on the display area 100.

In the present invention, while displaying the entire time series image frame in a vertically arranged map structure, a landmark for distinguishing the various organs belonging to the digestive organ 41 is further displayed. 3 shows an example of displaying the entire time series image frame, and displaying the image frames by color (eg, stomach, small intestine, and large intestine) of the digestive organs.

As an example of the landmark, the background color or the border color may be different for each part of the digestive organ 41 when the image frames are continuously arranged and displayed in time. Thus, it may indicate which organ of the digestive organ 41 corresponds to the image frame. As another example, a landmark may be displayed between two image frames corresponding to a point where an organ is changed in the digestive organ 41. As another example, a landmark may be displayed between an area in which the multiple time series image frames corresponding to the first organ of the digestive organ 41 are displayed and an area in which the multiple time series image frames corresponding to the second organ are displayed.

In addition, the present invention continuously displays the entire time series image frame in a vertically arranged map structure, and further displays an annotation addition indicator 130 indicating whether the annotation is added to the image frame to which the annotation is added. 4 shows an example of displaying an entire time series image frame and further indicating whether or not an annotation for each image frame is included.

As a further example, in the present invention, only the image frames to which the annotation is added among the entire time series image frames may be continuously displayed in the display area 100 in a continuous arrangement. To this end, the controller 33 may classify the image frames to which the annotation is added among all the time series image frames, and display only the classified image frames on the display area 100 in a vertically arranged map structure.

As another example, in the present invention, when displaying only the video frame to which the annotation is added while displaying the region representative video frame on the display area 100, annotation of multiple image frames belonging to the temporal section of each region representative video frame is added. It is preferable to further display whether or not the corresponding representative video frame. That is, if annotations are added to two image frames among the plurality of image frames belonging to the temporal interval of one interval representative image frame, the annotation additional indicators corresponding to the two annotations of the interval representative image frame may be further displayed. For example, an annotation is added to the B + 5th video frame among the B + 1, B + 2, B + 3, B + 4, and B + 5th video frames belonging to the temporal section of the B video frame, which is the representative video frame. If so, an indicator indicating whether an annotation of the B + 5th image frame is added to the Bth image frame may be further displayed.

The display unit 34 further provides an image sequence bar 110, a control bar 120, and an image frame selection icon 140 on the screen except for the display area 100. This is referred to FIGS. 3 to 7.

5 to 7 are diagrams showing enlarged display examples of time series image frames in a workstation of the present invention.

In particular, Figure 5 is to distinguish the image frame for each part of the digestive organs by color, and at the same time to further indicate whether or not to include an annotation for each image frame at the same time to display the minimum size ratio K times (K is a natural number) than in This is an example of displaying the enlarged ratio.

In addition, FIG. 6 is an example in which the image frames of the digestive organs are divided and displayed in color, but the L times (L is a natural number) is enlarged and displayed in FIG. 5, and FIG. This is an example of displaying the image at a maximum size ratio.

The control bar 120 is an image tool for adjusting the size of the multiple time series image frames displayed on the display area 100. The control bar 120 enlarges or reduces the size of the multiple time series image frames displayed on the display area 100. It has a first indicator used to. The controller 33 enlarges or reduces the plurality of time series image frames displayed on the display area 100 and displays them on the display area 100 as the first indicator moves.

In particular, the controller 33 sets an enlargement or reduction ratio of the plurality of time series image frames displayed on the display area 100 according to the movement of the first indicator, and the time series image to be displayed on the display area 100 according to the set ratio. The number of frames is determined, and the time series image frames are enlarged or reduced as much as the determined number to be displayed in a map structure arranged vertically in the display area 100. For example, the first indicator moves according to the manipulation of the user input unit 35, and may be moved by the rolling operation of the mouse wheel or by the page up / down of the keyboard or the pressing of a direction key.

If the first indicator of the control bar 120 is moved upward in FIG. 3 or 4, as shown in FIGS. 5 to 7, the image frame is enlarged and displayed on the display area 100 by a predetermined magnification ratio. .

Meanwhile, the number of time series image frames determined by the control unit 33 according to the enlargement or reduction ratio may be determined as follows. For example, when the number of total time series image frames is N * M (Column * Row), and the image is enlarged by K times, the controller 33 displays the time series images to be displayed on the display area 100 in inverse proportion to the enlargement ratio K times. Determine the number of frames. In this case, the time series image frame to be enlarged and displayed includes a plurality of previous image frames in time and a plurality of subsequent image frames in time, centering on one image frame determined among all time series image frames. In addition, one image frame which is centered in time may be one located at a spatial center in the time series image frames displayed on the current screen or any one selected through the user input unit 35.

For example, assuming that 77 time series image frames to be enlarged and displayed according to the movement of the first indicator and that the N + 100th image frame is spatially centered on the current screen are displayed, the display is enlarged among all the time series image frames. The video frame to be included includes the N + 62 th video frame from the N + 62 th video frame. When the enlarged and displayed image frame is determined, the controller 33 displays the enlarged N + 138 th image frame from the N + 62 th image frame in the display area 100, but displays the entire time series image frame. Rearranged and displayed in a vertically arranged map structure. This example can be understood as changing from the display example of FIG. 3 to the display example of FIG. 5.

As another example, assuming that the number of time series image frames to be enlarged and displayed according to the movement of the first indicator while the N + 50th image frame is selected through the user input unit 35 is determined to be 77, The video frame to be displayed includes the N + 12th video frame from the N + 88th video frame. When the enlarged and displayed image frame is determined, the controller 33 displays the enlarged N + 88th image frame from the N + 88th image frame on the display area 100, but displays the entire time series image frame. Rearranged and displayed in a vertically arranged map structure.

The image sequence bar 110 includes sequence bar indicators 111, 112, and 113 and indicates temporal lengths and temporal positions of the plurality of time series image frames displayed on the display area 100. In particular, the length of the image sequence bar 110 indicates the temporal length of the plurality of time series image frames displayed on the display area 100, and is displayed on the display area 100 by the positions of the sequence bar indicators 111, 112, and 113. Time position of multiple time series image frames.

In detail, the sequence bar indicators 111, 112, and 113 indicate a temporal position of the image frame displayed by being enlarged or reduced in size in the display area 100 compared to the entire time series image frame corresponding to the entire length of the image sequence bar 110.

The sequence bar indicators 111, 112, and 113 vary in width depending on the size of the plurality of time series image frames displayed in the display area 100. For example, the size of the plurality of time series image frames displayed on the display area 100 changes as the first indicator of the control bar 120 changes, and the sequence bar indicator changes according to the size change of the plurality of time series image frames. The width of (111, 112, 113) changes.

Accordingly, the sequence bar indicators 111, 112, and 113 may represent the amount (number) of the plurality of time series image frames displayed on the display area 100 according to an enlargement or reduction ratio as the width displayed on the image sequence bar 110.

Meanwhile, when the image frame is enlarged to the maximum size in the display area 100 according to the movement of the first indicator of the control bar 120, the controller 33 is enlarged to the maximum size as shown in FIG. 7. Control to display the time-series image frame on the display unit 34 in a column or to display on the display unit 34 in a row.

As shown in FIG. 7, when the image frame is enlarged to the maximum size in the display area 100, the controller 33 automatically displays icons 101 and 103 for sliding while moving to the previous or subsequent image frame in time. ) And a stop icon 102 for stopping the sliding movement is further displayed on the display area 100.

As such, when the image frame is enlarged and displayed in the display area 100 by a predetermined enlargement ratio, the number of time series image frames displayed in the display area 100 is reduced, and accordingly, the sequence bar indicator of the image sequence bar 120 ( 111, 112, and 113 are reduced in width. On the other hand, when the image frame is reduced and displayed in the display area 100 by a predetermined reduction ratio, the number of time series image frames displayed in the display area 100 increases, and accordingly, the sequence bar indicator of the image sequence bar 120. Reference numerals 111, 112, and 113 are enlarged in width.

When the image frame is enlarged and displayed in the display area 100 to the maximum as shown in FIG. 7, the number of time series image frames displayed in the display area 100 is minimized, and accordingly, the sequence bar indicator of the image sequence bar 120 is displayed. 111, 112, and 113 are displayed with their width reduced to the minimum width.

The present invention further provides an image frame selection icon 140 that is a control image for region selection of a part 141 of all time series image frames displayed on the display area 100.

FIG. 8 is a diagram illustrating an example of selecting a part of the entire time series image frame and setting a comment on the selected portion in the workstation of the present invention.

A tool for marking a condition on an image frame included in the area 141 selected by the image frame selection icon 140 shown in FIG. 8 may be provided, for which the control unit 33 includes a plurality of conditions. A tool for selecting a category may also be further provided via the display 34.

In addition, a tool for deleting an image frame included in the region 141 selected by the image frame selection icon 140 may be further provided. Accordingly, the controller 33 rearranges the remaining image frames except for the deleted image frames in the vertical and horizontal order in the time series while deleting the image frames included in the region 141 selected by the image frame selection icon 140. Control to display on the display area 100 in a map structure.

In particular, an example of the above-described image frame selection icon 140 includes a magic wand for selecting a region 141 including at least one image frame by a mouse click. There may be a pencil tool that draws and selects an area 141 including an image frame with a mouse cursor.

9 is a diagram illustrating an example of a mini-map for a selection area according to an embodiment of the present invention. When specific areas are selected by the image frame selection icon 140, the display shape of individual image frames is excluded. The mini-map displaying only the regions selected by the image frame selection icon 140 may be further displayed.

Here, the mini map may be displayed simultaneously with the display area 100 or only one of the two may be displayed in a toggle manner with the display area 100.

An example of generating the section representative video frame by the controller 33 will be described in detail below.

The control unit 33 preferably limits the number of interval representative image frames to be generated in the entire time series image frame. That is, in order to simplify tens of thousands of video frames on one screen, the representative video frame is generated by the number of sections that can be divided and displayed on the screen. If the example shown in FIG. 3 is an example of displaying an interval representative image frame, 14 * 22 (= 308) of all time series image frames are determined and generated as an interval representative image frame.

As another example, in the present invention, the section representative image frame may be generated by Y times the number of image frames that can be divided and displayed on one screen. Accordingly, the section representative image frame may be displayed over the Y screen pages. have. That is, if the example shown in FIG. 3 displays an interval representative video frame, 308 * 2 (= 616) of all time series video frames are selected and generated as the interval representative video frame. The 616 interval representative video frames can be divided into two screen pages for display.

Of course, when displaying the entire time series image frame, even if the total number of time series image frames is Y times the number of image frames that can be divided and displayed on one screen, the time series image frames may be displayed over Y screen pages. have. That is, if the example shown in FIG. 3 displays an entire time series image frame, the entire time series image frame may be divided into Y screen pages for display.

On the other hand, the interval representative video frame may be determined in the following manner.

First, a temporal section may be divided based on an image frame captured by a user among all time series image frames, and the captured image frame may be defined as a section representative image frame.

Second, the entire time series video frame may be divided into temporal sections having the same length, and the fastest video frame temporally in each temporal section may be defined as the section representative video frame.

Third, the temporal section may be divided based on the annotated video frame among the entire time series video frames, and the video frame to which the annotation is added may be defined as the section representative video frame. However, in this case, since there may be too many annotated video frames, when the annotated video frames are continuous in time, one of the temporally continuous video frames may be designated as the interval representative video frame.

For example, among the video frames to which an interval representative video frame is determined and annotated, the A, A + 1 and A + 2nd video frames, the A + 10th video frame, and the A + 20 and A + temporally. If an annotation is added to the 21st video frame, one of three video frames that are temporally continuous is first designated as the segment representative video frame, and then one of the two video frames that are temporally consecutive is designated as another segment representative video frame. .

Hereinafter, an image data processing procedure received from a capsule endoscope will be described.

10 is a flowchart illustrating an image data processing procedure according to the present invention, which is an image data processing procedure under the control of the controller 33.

Referring to FIG. 10, the controller 33 stores the received image data in the storage 32 (S1).

Subsequently, the controller 33 generates a plurality of time series image frames by reading the stored image data (S2). The multiple time series image frames are stored in the storage unit 32.

Subsequently, the controller 33 reads a plurality of stored time series image frames and continuously arranges them in a temporal manner and displays them in a map structure on the display unit 34 (S3).

FIG. 11 is a flowchart illustrating an enlargement or reduction process of a time series image frame according to an exemplary embodiment of the present invention. In accordance with the movement of an indicator included in the control bar 120, the control unit 33 enlarges or reduces the time series image frame. Is a display procedure.

Referring to FIG. 11, first, the control unit 33 reads a plurality of stored time series image frames and continuously arranges them in a temporal manner and displays them in a map structure on the display unit 34 (S11).

Subsequently, the controller 33 checks whether there is an enlargement or reduction command according to whether the indicator provided in the control bar 120 moves (S12).

If there is an enlargement or reduction command, the control unit 33 sets the enlargement or reduction ratio according to the movement of the indicator included in the control bar 120 (S13).

The controller 33 determines the number of image frames to be displayed on one screen (in detail, the display area 100) according to the set enlargement or reduction ratio (S14).

 Subsequently, the controller 33 performs an enlargement or reduction process on the determined number of image frames, and subsequently sequentially rearranges the enlarged or reduced image frames in time series and displays them on the display unit 34 in a map structure (S15). ).

Meanwhile, the present invention may display the interval representative image frame instead of the entire time series image frame, an example of which will be described with reference to FIG. 12.

FIG. 12 is a flowchart illustrating a processing procedure for displaying image data in a section representative image frame according to the present invention, and is an image data processing procedure under the control of the controller 33.

Referring to FIG. 12, the controller 33 stores the received image data in the storage 32 (S110).

Subsequently, the controller 33 generates a plurality of time series image frames by reading the stored image data (S120). The multiple time series image frames are stored in the storage unit 32.

Subsequently, the controller 33 divides the stored multiple time series image frames into a plurality of temporal sections (S130), and selects a section representative image frame representing each temporal section (S140). Here, the selected section representative image frame may be stored in the storage unit 32.

In addition, the controller 33 continuously arranges the selected section representative image frames in time and displays them on the display unit 34 in a map structure (S150).

The enlargement or reduction processing for the section representative image frame displayed in the map structure follows the above-described processing procedure of FIG. 11.

Subsequently, if automatic playback is required by selecting one of the displayed section representative video frames (S160), the controller 33 continuously and continuously outputs a plurality of video frames belonging to the temporal section of the selected section representative video frame at predetermined time intervals. Play back (S170).

In the workstation 30 of the present invention, the display unit 34 reduces and reduces the multiple time series image frames displayed on the display region 100 in addition to the display region 100 for continuously arranging and displaying the multiple time series image frames. The simplified mini map 200 may be further provided on the screen.

13 is a diagram illustrating an example of additionally displaying a mini map for a display area in the present invention.

The mini map 200 illustrated in FIG. 13 displays a reduced and simplified display of all time series image frames that can be displayed on the display area 100. Each image frame displayed on the mini map 200 is represented by a display area ( The size may be smaller or the resolution may be lower than that of each image frame displayed on 100).

In particular, as described above, in the display area 100, the image frame is enlarged or reduced by the control bar 120 and displayed.

However, in the mini map 200, the entire time series image frame is more simplified and displayed, but is not enlarged or reduced. That is, the mini map 200 represents the total time series image frames as fewer image frames than the total time series image frames, and since the mini map 200 displays the total time series image frames because the total number of time series image frames is displayed. A map structure with fewer rows or columns than the map structure of.

Meanwhile, in the present invention, the display area 100 and the mini map 200 are simultaneously provided on one screen, and the image frame displayed on the display area 100 corresponds to which position of the mini map 200. Can be represented. 13 illustrates an example in which the corresponding areas 201 are displayed in different colors.

 If the entire time series image frame is displayed in the display area 100 as shown in FIGS. 3 and 4, the mini map 200 corresponds to the entire time series image frame displayed in the display area 100. No indication is required.

However, when a part of the entire time series image frame is displayed in the display area 100 in an enlarged manner, the mini map (color, outline, etc.) may be used to determine which position of the enlarged image frame corresponds to the mini map 200. 200) is preferable.

In the present invention, a structure in which video frames are continuously arranged in time and displayed is shifted by one column in a horizontal column while the video frames are shifted by one row in a column or a column in which the video frames are arranged downward in each column. This corresponds to a structure of arraying rightward, and has been described as a structure of continuously arranging the map structure in the vertical order.

In addition, although the present invention has been described as a time series image frame according to a time sequence, a plurality of image frames in which the time series image frames are rearranged according to the spatial order within the digestion period during which the capsule endoscope moves are sequentially arranged according to the spatial order. It may be displayed on the screen. In addition, an image frame photographing an important part of the entire time series image frame may be preferentially arranged and displayed. That is, in the present invention, the image frame displayed on the screen is not limited to the image frame in time series.

While the preferred embodiments of the present invention have been described so far, those skilled in the art may implement the present invention in a modified form without departing from the essential characteristics of the present invention.

Therefore, the embodiments of the present invention described herein are to be considered in descriptive sense only and not for purposes of limitation, and the scope of the present invention is shown in the appended claims rather than the foregoing description, and all differences within the scope are equivalent to Should be interpreted as being included in.

10: capsule endoscope 20: receiver
30 workstation 31: receiver
32: storage unit 33: control unit
34: display unit 35: user input unit
40: human body 41: digestive system
100: display area 101: reverse direction icon
102: stop icon 103: forward progress icon
110: image sequence bar 111, 112, 113: sequence bar indicator
120: control bar 130: annotation annotation indicator
140: video frame selection icon 141: video frame selection area
200: mini map

Claims (20)

A capsule endoscope that photographs the inside of the digestive organ while moving along the digestive organ of the human body to generate an image signal, and continuously transmits the generated image signal;
A receiver for receiving and storing the image signal from a sensor that detects the image signal in the human body;
And a workstation including a receiving unit receiving the stored image signal from the receiver, a control unit processing the image signal received by the receiving unit, and a display unit displaying an image according to the processing result of the control unit on a screen.
The control unit,
Generating a plurality of image frames from the image signal received by the receiver; Displaying the image on the screen in which the plurality of generated image frames are continuously arranged; Setting an enlargement or reduction ratio of the plurality of image frames according to an enlargement or reduction command of the displayed image; Determining the number of multiple image frames to be displayed on the screen according to the set ratio; Enlarge or reduce the size of the determined number of image frames; Capsule endoscope system, characterized in that for controlling to display the enlarged or reduced image frame arranged on the screen continuously. The method of claim 1, wherein the display unit,
And a display area for continuously arranging and displaying the plurality of image frames, wherein the image sequence bar indicates a temporal length and a temporal position of the plurality of image frames displayed on the display area in addition to the display area. Capsule endoscope system, characterized in that further providing a control bar (control bar) for the enlargement or reduction of the plurality of image frames displayed on the display area on the screen. The display apparatus of claim 2, wherein the control bar includes a first indicator used to enlarge or reduce the size of the plurality of image frames displayed on the display area.
The control unit capsule endoscope system, characterized in that for setting the enlargement or reduction ratio of the plurality of image frames in accordance with the movement of the first indicator. The method of claim 3, wherein the number of multiple video frames determined according to the set ratio is:
And a plurality of previous image frames and a plurality of subsequent image frames in time more centered on one spatially centered or one selected by a user among all image frames displayed in the display area. Capsule Endoscopy System. The method of claim 2, wherein the image sequence bar includes a sequence bar indicator.
The sequence bar indicator is a capsule endoscope system, characterized in that the size is adjusted by the control bar relative to the entire image frame corresponding to the entire length of the image sequence bar to indicate the temporal position of the image frame displayed on the display area. The method of claim 5, wherein the width of the sequence bar indicator is changed according to the size of the image frame controlled by the control bar, characterized in that the width represents the amount of the image frame is displayed enlarged or reduced in the display area. Capsule endoscopy system. The method of claim 1, wherein the display unit,
And a display area for continuously arranging and displaying the plurality of image frames, and further providing an image frame selection icon on the screen to select a portion of the plurality of image frames displayed on the display area in addition to the display area. Capsule endoscopy system characterized in. The method of claim 1, wherein the control unit,
Capsule endoscope system, characterized in that for generating the plurality of video frames by adding an annotation for each video frame to the video signal received by the receiver. The capsule endoscope system according to claim 8, wherein the image frame to which the annotation is added among the plurality of image frames has an indicator indicating whether the annotation is added. The method of claim 9, wherein the control unit,
Capsule endoscope system, characterized in that to classify the image frame having the indicator among the plurality of image frames, and to arrange the sorted image frame on the screen. The method of claim 1, wherein the workstation,
And a storage unit for storing the video signal received by the receiver and the plurality of video frames generated by the controller. The method of claim 1, wherein the control unit,
Divide the generated plurality of image frames into a plurality of temporal sections,
Select a video frame representative of each temporal section,
The capsule endoscope system, characterized in that further control to display on the screen the image arranged in succession of the selected multiple-section representative image frame. The method of claim 12, wherein the control unit,
And when one section representative video frame displayed on the screen is selected, at least one video frame belonging to the temporal section of the selected section representative video frame is automatically reproduced at a predetermined time interval. The method of claim 1, wherein the display unit,
The plurality of image frames are continuously arranged and displayed on the screen in a map structure in a vertical order.
Capsules endoscope system, characterized in that the enlarged or reduced image frames are arranged in succession and displayed on the screen in a map structure of the vertical column. The map structure of claim 14, wherein the vertical order map structure comprises:
A capsule endoscope system, characterized in that it is formed by arranging image frames downward in each column while shifting the image frames by one column, or by arranging image frames in the right direction while shifting the image frames by one row. The method of claim 1, wherein the display unit,
A capsule endoscope system, comprising: a display area for continuously arranging and displaying the video frames; and further providing a mini map on the screen in which the image frame displayed on the display area is reduced and simplified in addition to the display area. . In the method for the workstation to process the image data received from the capsule endoscope,
Generating a plurality of image frames from the image signal received by the capsule endoscope;
Sequentially arranging the generated plurality of image frames and displaying them on a screen;
Setting an enlargement or reduction ratio of the displayed multiple image frames according to a command for adjusting the size of the displayed multiple image frames;
Determining the number of multiple image frames to be displayed on the screen according to the set ratio;
Enlarging or reducing the size of the determined number of image frames;
And rearranging and displaying the enlarged or reduced image frame on the screen continuously. 18. The capsule endoscope system according to claim 17, wherein after arranging the plurality of image frames or rearranging the enlarged or reduced image frames, the capsule endoscope system is displayed on the screen in a vertically arranged map structure. The method of claim 17, wherein the generated multiple image frame,
Capsule endoscopy system characterized in that the divided plurality of video frames divided into a plurality of temporal section after the interval representative image frame representing each of the selected temporal section. 20. The method of claim 19, wherein after the section representative video frame is displayed on the screen, if a particular section representative video frame is selected, a plurality of video frames belonging to the temporal section of the selected section representative video frame are continuously arranged at predetermined time intervals. The image data processing method further comprises the step of automatically playing.

Images