JP2015053617A - Image processing system, image processing method, program, and storage medium - Google Patents

Abstract

PROBLEM TO BE SOLVED: To facilitate the grasp of the depth of a stereo image generated in response to an instruction to change the depth.SOLUTION: A determination part 103 divides a photographic image photographed by a photographing unit 101 into a left-eye image and a right-eye image and stores them in a left-eye image storage unit 104 and a right-eye image storage unit 105. An image generator 107 generates a stereo image from the stored left-eye image and right-eye image. An output part 108 displays the stereo image on a monitor 109. The image generator 107 generates the stereo image by adjusting the left-eye image and right-eye image in response to the instruction to change the depth.

Description

  The present invention relates to image processing of X-ray stereo images.

  A conventional X-ray imaging apparatus displays an imaging direction by imaging a lead character arranged in an imaging area together with a subject or displays information on an image to display an imaging direction (for example, refer to Patent Document 1). ). Patent Document 2 discloses an invention in which depth depth measurement is performed on an X-ray stereo image and depth distance information is presented, but the presentation of the photographing direction is not mentioned.

JP 2007-215717 A Japanese Patent Laid-Open No. 61-253042

  However, it is difficult for the observer to grasp the depth of the stereo image only by looking at the stereo image generated according to the depth change instruction.

  An object of the present invention is to provide an image processing apparatus and method that can easily grasp the depth in a stereo image generated in response to a depth change instruction.

  The present invention has the following configuration as one means for achieving the above object.

  The image processing apparatus according to the present invention includes a storage unit that stores a captured image captured by the imaging unit into a left-eye image and a right-eye image, and the left-eye image and the right-eye image stored in the storage unit. A generating unit configured to generate a stereo image from the image; and a display unit configured to display the stereo image, wherein the generating unit adjusts the left-eye image and the right-eye image according to a depth change instruction. The stereo image is generated.

  Further, a storage unit that stores the captured image captured by the imaging unit into a left-eye image and a right-eye image, and generates a stereo image from the left-eye image and the right-eye image stored in the storage unit. The image processing apparatus includes generation means, additional image generation means for generating an additional image including depth information of the stereo image, and display means for displaying the stereo image together with the additional image.

  According to the present invention, the depth can be easily grasped in the stereo image generated in response to the depth change instruction.

FIG. 2 is a block diagram illustrating a configuration example of the X-ray stereo image photographing apparatus according to the first embodiment. 3 is a flowchart for explaining image processing in the X-ray stereo image photographing apparatus according to the first embodiment. FIG. 3 is a block diagram illustrating a configuration example of an X-ray stereo image capturing apparatus according to a second embodiment. 9 is a flowchart for explaining image processing in the X-ray stereo image capturing apparatus according to the second embodiment. The flowchart explaining the determination process of the annotation information which an annotation information determination part performs. The figure explaining the method of determining the display size of a content according to the display position of a depth direction. FIG. 4 is a block diagram illustrating a configuration example of an X-ray stereo image capturing apparatus according to a third embodiment. 9 is a flowchart for explaining image processing in the X-ray stereo image photographing apparatus according to the third embodiment. The figure explaining the annotation display position when the first distance and the second distance are designated. FIG. 6 is a block diagram showing a configuration example of an X-ray stereo image photographing apparatus of Example 4. 10 is a flowchart for explaining image processing in the X-ray stereo image capturing apparatus of the fourth embodiment. The figure explaining a mode that the display method of a stereo image is changed. The figure which shows the example which displays the name and position of each site | part as annotation. FIG. 6 is a block diagram showing a configuration example of an X-ray stereo image photographing apparatus according to Embodiment 5. The flowchart explaining the determination process of the annotation information which an annotation information determination part performs.

  Hereinafter, image processing according to an embodiment of the present invention will be described in detail with reference to the drawings.

  The block diagram of FIG. 1 shows a configuration example of an X-ray stereo image photographing apparatus 100 that is an image processing apparatus of the first embodiment. In addition, image processing in the X-ray stereo image photographing apparatus 100 of the first embodiment will be described with reference to the flowchart of FIG.

  The receiving unit 102 receives a captured image including the left-eye image and the right-eye image captured by the capturing unit 101 in units of frames (S201). Note that the relationship between the imaging unit 101 and the receiving unit 102 may be a pair of the imaging unit 101a and the receiving unit 102a, or the imaging unit 101a of the first image, the first receiving unit 102a, and the second image. In some cases, the imaging unit 101b and the second receiving unit 102b are combined. The imaging unit 101 is an X-ray image imaging apparatus that performs imaging using one or more X-ray generators and an X-ray detector.

  When the receiving unit 102 receives a captured image (S202), the determining unit 103 determines whether the received captured image is a left-eye image or a right-eye image. Then, the left-eye image and the right-eye image are classified, and the left-eye image is stored in the memory of the left-eye image storage unit 104, and the right-eye image is stored in the memory of the right-eye image storage unit 105 (S204, S205). The determination of whether the image is for the left eye or the right eye is made based on, for example, information added to the header of the captured image.

  The output unit 108 outputs to the monitor 109 a stereo image that enables stereoscopic viewing of the captured image stored in the memory of the left-eye image storage unit 104 and the memory of the right-eye image storage unit 105 (S209). As long as there are subsequent frames based on the determination in step S210, the processes in steps S201 to S209 are repeated.

  When the image for the left eye is projected to the left eye of the observer and the image for the right eye is projected to the right eye of the observer, the observer can perform correct stereoscopic vision. If the image for the right eye is projected to the left eye of the observer and the image for the left eye is projected to the right eye of the observer, the viewer's recognition of the stereo image is perceived in the front side by reversing the front and back depth. Is perceived on the far side and what is originally perceived on the far side is perceived on the near side.

  For example, when a depth reversal occurs in a computer graphics (CG) image, a stereo image that is different from the intention of the CG image is recognized, such as what is supposed to be perceived as protruding and perceived as being recessed. Usually, the observer feels uncomfortable. However, since the X-ray image is a transmission image, the observer does not feel uncomfortable even if the depth is reversed. If depth inversion occurs in the X-ray image, a stereo image as seen from the back side of the X-ray detector is recognized.

  The image generation unit 107 determines the state of a flag indicating a change in depth instructed by the depth instruction unit 106 (S206). Then, according to the determination result of the depth change flag indicating the change instruction, the captured image of the same frame is read from the memory of the left-eye image storage unit 104 and the memory of the right-eye image storage unit 105, and these captured images are output to the output unit 108. To supply.

  When the depth change flag is off, the image generation unit 107 uses the captured image read from the memory of the left-eye image storage unit 104 as the left-eye image and uses the captured image read from the memory of the right-eye image storage unit 105 as the right-eye image. To the output unit 108 (S207). That is, normal supply is performed without changing the left and right.

  When the depth change flag is on, the image generation unit 107 uses the captured image read from the memory of the left-eye image storage unit 104 as the right-eye image and uses the captured image read from the memory of the right-eye image storage unit 105 as the left-eye image. To the output unit 108 (S208). In this way, the data reading method is changed to perform supply with the left and right being switched. That is, in response to the depth change instruction, the image generation unit 107 generates a stereo image using the captured image divided into the left-eye image as the right-eye image and the captured image divided into the right-eye image as the left-eye image. To do.

  Further, the image generation unit 107 may invert the left-eye image and the right-eye image in the baseline length direction (horizontal direction) instead of performing data exchange in step S208, and the same result is obtained. It is done. The baseline length direction refers to the direction in which two X-ray sources are arranged. That is, the image generation unit 107 inverts each of the left-eye image and the right-eye image in the horizontal direction in response to the depth change instruction, and generates a stereo image. In this way, the user can reverse the sense of depth before and after in the stereo image by operating the depth instruction unit 106.

  Note that the image generation unit 107 can also generate a stereo image by adjusting the overlapping degree of the left-eye image and the right-eye image in accordance with a depth change instruction. Specifically, the image generation unit 107 generates a stereo image so that the overlapping region of the right-eye image with respect to the left-eye image is widened in response to an instruction to increase the depth. The image generation unit 107 generates a stereo image so that the overlapping region of the right-eye image with respect to the left-eye image is narrowed in response to an instruction not having depth.

  The depth instruction unit 106 is a switch or button arranged on the operation panel of the X-ray stereo image photographing apparatus 100, or a button arranged on a user interface (UI) displayed on the monitor 109, for example.

  As described above, according to the image processing apparatus of the present invention, the storage means (the left-eye image storage unit 104, the right-eye storage unit) stores the captured image captured by the imaging unit (imaging unit 101) by dividing it into a left-eye image and a right-eye image. Image storage unit 105) and generation unit (image generation unit 107) that generates a stereo image from the left-eye image and right-eye image stored in the storage unit (left-eye image storage unit 104, right-eye image storage unit 105). And a display unit (monitor 109) for displaying a stereo image, and the generation unit (image generation unit 107) adjusts the left-eye image and the right-eye image in accordance with the depth change instruction, Generate.

  Therefore, the observer can easily grasp the depth in the stereo image generated according to the depth change instruction.

  The image processing according to the second embodiment of the present invention will be described below. Note that the same reference numerals in the second embodiment denote the same parts as in the first embodiment, and a detailed description thereof will be omitted.

  In stereo display, stereoscopic vision is enabled by setting parallax between the left-eye image and the right-eye image. Therefore, the annotation can be stereoscopically viewed by setting parallax in the annotation. The annotation is additional information such as characters and symbols displayed on the image. When displaying depth information such as the X-ray incident direction and subject orientation, the annotation is displayed in three dimensions so that additional information corresponding to the near side is recognized on the near side and additional information corresponding to the far side is recognized on the far side. It is considered that information can be presented more easily when viewed.

  A block diagram of FIG. 3 shows an example of the configuration of the X-ray stereo image photographing apparatus 100 of the second embodiment. Also, image processing in the X-ray stereo image photographing apparatus 100 of the second embodiment will be described with reference to the flowchart of FIG.

  The X-ray stereo image capturing apparatus 100 according to the second embodiment has a configuration for realizing a function of generating an annotation and adding the generated annotation to a captured image. First, the annotation information determination process (S401) executed by the annotation information determination unit 111 will be described with reference to the flowchart of FIG.

  The annotation information determination unit 111 inputs additional information (hereinafter, content) such as characters and symbols to be displayed as an annotation (S501). Note that the content is input via the operation unit of the X-ray stereo image capturing apparatus 100 or the UI of the monitor 109.

  Next, the annotation information determination unit 111 determines display conditions for the content. That is, the two-dimensional display position is determined (S502), the depth display position is determined (S503), and the display size such as the character size is determined (S504). The display size determined here is a size (specified size) perceived on the display when there is no parallax. The display size that is actually perceived three-dimensionally is set by the following method so that the front side is large and the back side is small according to the display position of the depth so that there is no sense of incongruity in the depth sensation. .

A method for determining the display size of content according to the display position of the depth will be described with reference to FIG. Assume that the position of the eyeball 601 is (Xe, Ye, Ze), and the display position 602 of the content on the display surface is (Xp, Yp, 0). When content is perceived at a position 603 that is a distance L away from the display surface in the depth direction, the size D calculated by the following formula for the specified size (content size perceived when focused on the display surface) Generate a small annotation display.
D = L (Xe-Xp) / Ze (1)

  Note that the display size of each content can be arbitrarily specified by the user.

  Next, the annotation image generation unit 112 as an additional image generation unit generates an annotation image as an additional image based on the annotation information determined by the annotation information determination unit 111 (S402). The annotation image includes a left-eye annotation image (hereinafter referred to as a left-eye additional image) and a right-eye annotation image (hereinafter referred to as a right-eye additional image). The left-eye additional image is stored in the memory of the left-eye annotation management unit 113, and the right-eye additional image is stored in the memory of the right-eye annotation management unit 114.

  The subsequent processing (S201 to S205) from reception of the captured image to storage of the captured image is the same as in the first embodiment.

  Next, the left-eye image combining unit 115 generates a left-eye composite image by combining the left-eye image and the left-eye additional image (S403). Further, the right-eye image combining unit 116 generates a right-eye combined image by combining the right-eye image and the right-eye additional image (S404). Then, the image composition unit 117 generates a final stereo image obtained by compositing the left-eye composite image and the right-eye composite image, and supplies the stereo image to the output unit 108 (S405).

  The order of synthesis is not limited to the above. In other words, a stereo image is generated by combining the left-eye image and the right-eye image, and an annotation image is generated by combining the left-eye additional image and the right-eye additional image. There is no difference in the typical processing results.

  For example, when the X-ray incidence direction when X-rays are incident from the front toward the back is indicated by annotation, the character string `` In '' is perceived at the front and the character string `` Out '' is perceived at the back, X-ray incident direction is displayed in three dimensions. Also, when the shooting direction when shooting AP images (front and back images) is indicated by annotation, the shooting direction is displayed in three dimensions so that the letter “A” is perceived in front and the letter “P” is perceived in the foreground. To do.

  As described above, according to the image processing apparatus of the present invention, the storage means (the left-eye image storage unit 104, the right-eye storage unit) stores the captured image captured by the imaging unit (imaging unit 101) by dividing it into a left-eye image and a right-eye image. Image storage unit 105) and generation unit (image generation unit 107) that generates a stereo image from the left-eye image and right-eye image stored in the storage unit (left-eye image storage unit 104, right-eye image storage unit 105). And additional image generation means (annotation image generation unit 112) for generating an additional image including depth information, and display means (monitor 109) for displaying a stereo image together with the additional image.

  Also, a synthesis means (left-eye image synthesis unit 115, which generates a left-eye synthesized image obtained by synthesizing the left-eye additional image with the left-eye image and a right-eye synthesized image obtained by synthesizing the right-eye added image with the right-eye image. And a generating means (image combining unit 117) that combines the left-eye combined image and the right-eye combined image to generate a stereo image.

  An observer who observes the annotation stereoscopically can instantaneously determine the incident direction of the X-ray, the direction of the subject, and the like from the sense of depth of the annotation.

  Hereinafter, image processing according to the third embodiment of the present invention will be described. Note that the same reference numerals in the third embodiment denote the same parts as in the first and second embodiments, and a detailed description thereof will be omitted.

  In the third embodiment, an example in which annotation information is automatically generated and an annotation display position is set according to a preset imaging method (imaging site, imaging direction, etc.) will be described.

  A block diagram of FIG. 7 shows a configuration example of the X-ray stereo image photographing apparatus 100 of the third embodiment. Further, the image processing in the X-ray stereo image photographing apparatus 100 of the third embodiment will be described with reference to the flowchart of FIG.

  Prior to the determination of annotation information, the imaging method input unit 121 inputs information on an imaging method indicating an imaging region and an imaging direction (S701). Note that the imaging method information is input via the operation unit of the X-ray stereo image capturing apparatus 100 or the UI of the monitor 109. The annotation information determination unit 111 automatically determines annotation information based on the shooting method input by the shooting method input unit 121 and a preset shooting method (S702). The subsequent processing is the same as the processing after step S402 shown in FIG.

  Further, the annotation image generation unit 609 can set the display position of the content in accordance with the designation of the first distance and the second distance input by the shooting method input unit 121 or the annotation information determination unit 111. For example, the distance from the X-ray source to the subject can be designated as the first distance, and the distance from the X-ray source to the X-ray sensor can be designated as the second distance.

  The annotation display position when the first distance and the second distance are designated will be described with reference to FIG. That is, the character string “IN” is displayed so as to be perceived at the first distance L1, and the character string “OUT” is displayed so as to be perceived at the second distance L2. The parallax to make the character string perceived at the specified distance is the baseline length between the stereo X-ray source at the time of imaging, the distance between the X-ray source and the X-ray detector, the distance between the eyeball and the display at the time of observation, It is calculated from the relationship between the display size, pixel pitch, X-ray detector size, and pixel pitch.

  As described above, the additional information is automatically determined in accordance with a preset photographing method, so that it is possible to reduce time and effort for the user to input information related to the additional information. Furthermore, by inputting an instruction for the display position of the depth of the content, an annotation display that is easier for the observer to observe is possible.

  Hereinafter, image processing according to the fourth embodiment of the present invention will be described. Note that the same reference numerals in the fourth embodiment denote the same parts as in the first to third embodiments, and a detailed description thereof will be omitted.

  As described in Example 1, when reversing the sense of depth in the stereo image viewed from the observer, the sense of depth of the annotated content is also reversed, so that the stereo image, the sense of depth, and the sense of depth of the content are reversed. Must match. If the left-eye image and the right-eye image are simply interchanged, small content is perceived in front and large content is perceived in the back. In the fourth embodiment, an example in which the depth sensation of content is reversed together with the depth sensation of a stereo image will be described.

  A block diagram of FIG. 10 shows a configuration example of the X-ray stereo image photographing apparatus 100 of the fourth embodiment. Further, image processing in the X-ray stereo image photographing apparatus 100 of the fourth embodiment will be described with reference to the flowchart of FIG. In FIG. 11, processes that are substantially the same as the processes of the second embodiment shown in FIG.

  When annotation information is determined by the annotation information determination unit 111 (S401), the annotation image generation unit 112 and the image composition unit 117 acquire the state of the depth change flag set by the depth instruction unit 106 (S801). As in the third embodiment, the information on the shooting method may be input and the annotation information may be determined based on the shooting method, and then the state of the depth change flag may be acquired.

  Next, the annotation image generation unit 112 generates an annotation image based on the annotation information determined by the annotation information determination unit 111 and the state of the depth change flag (S802). That is, when the depth change flag is off, an annotation image is generated according to the annotation information. When the depth change flag is on, the annotation image is generated by switching the display size indicated by the annotation information between the left-eye additional image and the right-eye additional image.

  Subsequently, after processing similar to that in the second embodiment is performed in steps S201 to S404, the image composition unit 117 determines the state of the acquired depth change flag (S803). Then, according to the determination result, a stereo image is generated by combining the composite image supplied from the left-eye image combining unit 115 and the composite image supplied from the right-eye image combining unit 116.

  When the state of the depth change flag is off, the image composition unit 117 uses the composite image supplied from the left-eye image composition unit 115 as the left-eye image, and uses the composite image supplied from the right-eye image composition unit 116 as the right-eye image. Is synthesized (S804). In other words, normal synthesis is performed in which the left and right are not interchanged.

  On the other hand, when the depth change flag is on, the image composition unit 117 uses the composite image supplied from the left-eye image composition unit 115 as the right-eye image, and uses the composite image supplied from the right-eye image composition unit 116 as the left-eye image. Is synthesized (S805). In other words, the left and right are combined by exchanging data.

  Similarly to the first embodiment, the image composition unit 117 may invert the left-eye image and the right-eye image in the baseline length direction in step S805, instead of performing data replacement, Results are obtained.

  The manner in which the stereo image display method is changed will be described with reference to FIG. FIG. 12 shows an example in which the X-ray incident direction is displayed as an annotation, and FIG. 12 (a) is added with an annotation display in which X-rays are irradiated from the front to the back. FIG. 12B shows a case where the display method of the stereo image is changed by the depth change flag, and an annotation display in which X-rays are irradiated from the back to the front is added.

  In the fourth embodiment, when it is determined in step S210 that there is a next frame, the process returns to step S801, and the depth change flag state is acquired. If there is a change in the state of the depth change flag, the annotation image generation unit 112 regenerates the annotation image in step S802. However, if there is no change in the state of the depth change flag, the annotation image generation unit 112 does not regenerate the annotation image.

  Thus, when the stereo image display method is changed and the depth sensation is reversed, the annotation image is also changed in accordance with the change of the stereo image display method. Therefore, the observer can easily determine the X-ray incident direction and imaging direction from the annotation display.

  Hereinafter, image processing according to the fifth embodiment of the present invention will be described. Note that the same reference numerals in the fifth embodiment denote the same parts as in the first to fourth embodiments, and a detailed description thereof will be omitted.

  FIG. 13 shows an example in which the name and position of each part are displayed as annotations. FIGS. 13 (a) and 13 (b) are examples in which, for example, the names and positions of the heart and spine in the case of AP imaging of the chest are displayed as annotations. In that case, it is desired to display the annotations of “heart” and “spine” in accordance with the depth positions where the corresponding parts are perceived.

  A block diagram of FIG. 14 shows a configuration example of the X-ray stereo image photographing apparatus 100 of the fifth embodiment. In FIG. 14, the same reference numerals are given to the processes that are substantially the same as the processes of the second embodiment shown in FIG. Further, the annotation information determination process executed by the annotation information determination unit 111 will be described with reference to the flowchart of FIG.

  When the annotation information determination unit 111 determines annotation information (S401), the annotation information determination unit 111 inputs designation of a target part for annotation display (S500). The depth information calculation unit 131 calculates the depth information of the target part for annotation display. The annotation information determination unit 111 executes determination of the depth display position (S503) and determination of the display size (S504) based on the calculation result of the depth information. The subsequent processing is the same as the processing of the second embodiment shown in FIG.

  Thus, since the names and positions of the respective parts are three-dimensionally annotated, the user can instantly recognize the three-dimensional positional relationship of the respective parts.

[Other Examples]
The present invention can also be realized by executing the following processing. That is, software (program) that realizes the functions of the above-described embodiments is supplied to a system or apparatus via a network or various recording media, and a computer (or CPU, MPU, etc.) of the system or apparatus reads the program. It is a process to be executed.

Claims (18)

Storage means for storing a photographed image photographed by the photographing means separately into a left-eye image and a right-eye image;
Generating means for generating a stereo image from the left-eye image and the right-eye image stored in the storage means;
Display means for displaying the stereo image,
The image processing apparatus, wherein the generation unit generates the stereo image by adjusting the left-eye image and the right-eye image according to a depth change instruction.   In response to the depth change instruction, the generation unit uses the captured image divided into the left-eye image as the right-eye image, and the captured image divided into the right-eye image as the left-eye image. 2. The image processing apparatus according to claim 1, wherein the image processing apparatus generates an image.   2. The generation unit according to claim 1, wherein the generation unit generates the stereo image by inverting each of the left-eye image and the right-eye image in a horizontal direction in response to the depth change instruction. Image processing device.   2. The image processing according to claim 1, wherein the generation unit generates the stereo image by adjusting an overlapping degree of the left-eye image and the right-eye image in accordance with the depth change instruction. apparatus. Storage means for storing a photographed image photographed by the photographing means separately into a left-eye image and a right-eye image;
Generating means for generating a stereo image from the left-eye image and the right-eye image stored in the storage means;
Additional image generation means for generating an additional image including depth information of the stereo image;
An image processing apparatus comprising: display means for displaying the stereo image together with the additional image. Further, the image processing apparatus further includes a combining unit that generates a composite image for the left eye that combines the image for the left eye with the image for the left eye, and a composite image for the right eye that combines the additional image for the right eye with the image for the right eye,
6. The image processing apparatus according to claim 5, wherein the generation unit combines the left-eye composite image and the right-eye composite image to generate a stereo image. Furthermore, an input means for inputting information on the photographing method of the photographed image,
7. The image processing apparatus according to claim 5, further comprising: a determination unit that determines additional information to be included in the additional image and display conditions of the additional image based on information on the photographing method.   8. The determining means determines content to be added to the stereo image as the additional information, and determines a two-dimensional display position, a depth display position, and a display size of the content as the display conditions. The image processing apparatus described in 1.   The additional image generating means generates the additional image so that the content corresponding to the two-dimensional display position, the depth display position, and the display size is perceived when the stereo image is observed. The image processing device according to claim 8.   9. The additional image generation unit changes the generation method of the additional image according to a depth change instruction, and the generation unit changes the generation method of the stereo image according to the depth change instruction. An image processing apparatus according to claim 9.   11. The image processing according to claim 10, wherein when the depth change is instructed, the additional image generation unit swaps the display size between the left-eye additional image and the right-eye additional image. apparatus.   When the depth change is instructed, the generation unit generates the stereo image using the left-eye composite image as the right-eye composite image and the right-eye composite image as the left-eye composite image. 12. The image processing device according to claim 10 or claim 11.   10. When the depth change is instructed, the generation unit generates the stereo image after performing a horizontal inversion process on each of the left-eye composite image and the right-eye composite image. 13. The image processing device according to claim 12.   The depth change instruction is an instruction for reversing the depth sensation of the stereo image when observing the stereo image. The image processing device described in any one of the above. Dividing the captured image into a left-eye image and a right-eye image and storing them;
Generating the stereo image by adjusting the left-eye image and the right-eye image in response to a depth change instruction;
And a step of displaying the stereo image. Dividing the captured image into a left-eye image and a right-eye image and storing them;
Generating a stereo image from the stored left-eye image and right-eye image;
Generating an additional image including depth information of the stereo image;
And a step of displaying the stereo image together with the additional image.   15. A program for causing a computer to function as each unit of the image processing apparatus according to any one of claims 1 to 14.   A computer-readable recording medium on which the program according to claim 17 is recorded.

Images