JP6765130B2 - Image generator - Google Patents

Description

The present invention relates to an apparatus for generating an image used for diagnosis and the like.

Currently, in addition to palpation and inspection, diagnosis (interpretation) using a 2D (two-dimensional) contour image by a moire camera is performed for the diagnosis of scoliosis.

In recent years, attention has been focused on 3D (three-dimensional) scanners that digitize the three-dimensional shape of an object as it is. In particular, the 3D body scanner can acquire the three-dimensional shape of the entire human body. Unlike CT examinations, 3D scanners cannot measure the state inside the human body, but they have the characteristic of being able to accurately measure the outer shape without exposure. Patent Document 1 discloses a system that generates information useful for diagnosing symptoms such as scoliosis from the surface shape of the human body measured by a 3D scanner and presents it to a diagnostician.

The evaluation system for spinal kyphosis disclosed in Patent Document 1 includes a three-dimensional data acquisition unit that photographs the back of a subject and acquires the three-dimensional data, and the three-dimensional data acquired by the three-dimensional data acquisition unit. The center line detection unit that detects the vertical center line of the back of the subject, the feature site designation section that specifies the feature site for measuring the degree of curvature of the back of the subject, and the feature site designation section. With respect to the designated feature part, the uneven state detection part that detects the uneven state of the body surface in the horizontal direction based on the three-dimensional data, and the feature part designated by the feature part designation part, with the center line as a boundary. It is provided with a left-right difference calculation unit for calculating the height difference of the peak positions of the left and right unevenness, and a display unit for displaying the detection result by the unevenness state detection unit and the calculation result by the left-right difference calculation unit.

Further, the contour image generation method for performing scoliosis screening disclosed in Patent Document 2 includes a step of capturing a back image of a subject and outputting a digital image, and a brightness signal for each pixel from the captured digital image. And the step of generating a quantized image obtained by quantizing the extracted brightness signal with a predetermined threshold reference, and the pixel of the same quantization level for each quantization level in the quantized image. It includes a step of converting into a connected image and a step of displaying an image in which pixels of the same quantization level are connected on a display so that a high degree of deviation on the back surface of the subject can be observed.

Japanese Patent No. 6132354 Japanese Patent No. 5652883

Since the conventional moire camera generates a 2D contour image based on the observation direction of the camera, it is necessary to make the posture of the evaluation target person (that is, the subject) face the camera at the time of shooting. However, the contour lines generated by the moire camera are greatly affected by the slight movements and postures of the evaluation subjects. In addition, since contour lines from different viewpoints cannot be regenerated for the image once taken, if an appropriate contour image cannot be obtained, it is necessary to take the picture again.

Since the information on the uneven state and the height difference displayed by the system of Patent Document 1 looks significantly different from the contour image generated by the moire camera, it takes time for the diagnostician to understand the information.

The contour image generated by the method of Patent Document 2 can be made to look similar to the contour image generated by the moire camera, but the height is increased by the method of connecting pixels of the same quantization level of the brightness signal. It is considered that the accuracy is not high.

The present invention can generate an image displaying a contour image of the back surface, is more convenient than a moire camera, and provides a contour image with higher accuracy than a method of generating a contour image based on the quantization level of a brightness signal. Provide possible equipment.

In one aspect, the present invention represents a means for acquiring a texture image and three-dimensional shape data of the back surface obtained by photographing the back surface of an evaluation target person, and a contour line of the back surface from the three-dimensional shape data. A generation means for generating an contour image, a synthesis means for generating a composite image by synthesizing the contour image generated by the generation means and the texture image, and an output means for outputting the composite image generated by the synthesis means. An image generator including, is provided.

In one embodiment, the generation means generates contour images of the back surface for each of the plurality of different observation directions from the three-dimensional shape data, and the synthesis means generates the plurality of observation directions. For each of the above, a composite image is generated by synthesizing the contour image and the texture image in the observation direction.

Another aspect further includes, for each of the plurality of observation directions, a virtual texture generation means for generating a texture image representing the back surface when virtually viewed from the observation direction from the texture image, and the composition thereof. The means generates a composite image in which the contour image and the texture image for the observation direction are combined for each of the plurality of observation directions.

In still another aspect, the output means is a means for displaying the composite image for each of the plurality of observation directions generated by the synthesis means on the screen, and a user among the composite images for each of the observation directions. A means for accepting selection and a means for registering the composite image selected by the user in a predetermined database in association with the identification information of the evaluation target person are included.

In still another aspect, the generation means generates a contour image from the three-dimensional shape data when the observation direction is a second direction different from the shooting direction when the back surface of the evaluation target person is photographed. Then, the synthesizing means synthesizes the contour image and the texture image when the second direction is the observation direction.

In yet another aspect, the compositing means further includes, from the texture image, a virtual texture generating means for generating a texture image when the back surface is virtually viewed from the second direction, which is the observation direction. , A composite image is generated by combining the contour image and the texture image in the second direction.

In yet another aspect, the background portion of the texture image that is the background of the back surface is specified based on the three-dimensional shape data, and the image of the background portion of the texture image can be distinguished from the back surface. The compositing means further includes a processing means for processing such as the above, and the compositing means synthesizes the texture image processed by the processing means and the contour image to generate the composite image.

According to the present invention, it is possible to provide a device that is more convenient than a moire camera and can generate an image displaying a contour image of the back surface.

It is a figure which illustrates the functional structure of the system of Embodiment 1. It is a figure for demonstrating the image generated by the system of embodiment. It is a figure for demonstrating an example of the form of the contour image. It is a figure for demonstrating another example of the form of a contour image. It is a figure for demonstrating another example of the form of a contour image. It is a figure for demonstrating another example of the form of a contour image. It is a figure which illustrates the functional structure of the system of Embodiment 2. It is a figure for demonstrating the contour image about the front direction of the real scanner (3D measuring apparatus). It is a figure for demonstrating the contour image about the observation direction different from the front direction of the real scanner (3D measuring apparatus). It is a figure which shows the example of the list screen of the composite image provided by the system of Embodiment 2. It is a figure which illustrates the functional structure of the system of Embodiment 3. It is a figure for demonstrating the composition of the texture image after transformation and the contour line image in Embodiment 3. FIG. It is a figure which illustrates the functional structure of the system of Embodiment 4.

<Embodiment 1>
FIG. 1 illustrates a functional configuration of an embodiment of a back-diagnosis image generation system according to the present invention. As shown in the figure, this system includes a 3D measuring device 10, a data separation unit 12, a contour image generation unit 14, an image composition unit 16, and an output unit 18.

The 3D measuring device 10 is a device that measures the 3D shape of the back surface of the evaluation target person (for example, the patient to be diagnosed). There are several 3D measurement methods, such as those based on the principle of triangulation (for example, stereo method, optical cutting method, phase shift method), and those based on TOF (Time of Flight). The 3D measuring device 10 may use any of the methods.

Preferably, as the 3D measuring device 10, a device capable of photographing the 3D shape data of the subject and the texture image with substantially the same optical axis and angle of view is used. For example, a 3D measuring device that uses a method such as a stereo method, an optical cutting method, or a phase shift method often satisfies this condition. The texture image is an image obtained by capturing the reflection of natural light or illumination light from the subject, and a typical example thereof is a color or monochrome image captured by a general camera. The 3D measuring device 10 outputs the 3D shape data and the texture image obtained by photographing. The 3D shape data is represented, for example, in the form of a distance image having a value of the depth (distance) z of each pixel in the image.

A camera for capturing a texture image of the evaluation target person may be prepared separately from the 3D measuring device 10. However, in this case, it is assumed that the requirements such as the three-dimensional position, the optical axis direction, and the angle of view of the camera and the 3D measuring device 10 are known. If these requirements are known, the texture image taken by the camera and the 3D shape data generated by the 3D measuring device 10 can be converted into a state of the same optical axis and the same angle of view and combined.

The data separation unit 12 separates the 3D shape data and the texture image input from the 3D measurement device 10, and inputs the 3D shape data to the contour image generation unit 14 and the texture image to the image composition unit 16. In the data separation unit 12, the 3D measuring device 10 is composed of output data in a data format in which a three-dimensional shape data and a texture image are integrated (for example, four-dimensional coordinates composed of RGB values of pixels and Z coordinates). It is necessary when outputting the image data). If the 3D measuring device 10 can output the 3D shape data and the texture image separately, the data separating unit 12 may be omitted.

The contour image generation unit 14 generates a contour image from the 3D shape data. The contour line image is an image showing contour lines connecting points (pixels) of the same height in the 3D shape represented by the 3D shape data at regular pitches (height intervals). Here, the height is defined as a plane in which an arbitrary observation position and observation direction are set for the 3D shape, and the plane passes through the observation position and has the observation direction as the normal, and the plane and the 3D shape are each 3D. It is the vertical distance from the coordinate point. A simple example is the depth distance z of each pixel when the 3D shape data is represented in the form of a distance image. The moire pattern generated by a conventional moire camera is also an example of an image representation showing contour lines.

The image synthesizing unit 16 synthesizes the contour line image generated by the contour line image generation unit 14 and the texture image taken by the 3D measuring device 10 to generate a composite image. The generated composite image is used as an evaluation image for evaluating the state of the back surface of the evaluation target person. For example, using a composite image, evaluation or diagnosis of whether or not the evaluation subject corresponds to scoliosis is performed.

The output unit 18 outputs the composite image generated by the image synthesis unit 16. This output is, for example, a form of displaying on the screen of an attached display device, a form of printing out from a printer, a form of registering in the diagnostic information database in association with the ID (patient number, etc.) of the evaluation target person, and the like. Perform in one or more forms.

The image composition by the image composition unit 16 will be described with reference to the specific example shown in FIG. In FIG. 2, the texture image 100 is an image showing the back surface of the evaluation target person, and the contour line image 102 is an image showing the contour lines of the 3D shape of the back surface of the evaluation target person at equal pitches. The illustrated contour image 102 is an image in which the value of the pixel on the contour line is "1" (that is, the maximum density of black) and the value of the pixel at a position other than the contour line is "0" (that is, colorless and transparent). In the figure, the black line indicates the contour line 104.

The image synthesizing unit 16 generates a composite image 110 by superimposing the contour image 102 on the texture image 100 in pixel units. In this example, the contour line 104 indicated by the contour line image 102 is written on the texture image 100, and the pixel value of the texture image 100 is valid as it is for the pixels other than the contour line.

The contour image 102 illustrated in FIG. 2 is merely an example. Hereinafter, various forms of the contour image will be described with reference to FIGS. 3 to 7. In these figures, the coordinate axes of the contour lines in the observation direction (that is, the height direction) are Z, and the pitch of the contour lines is p. Further, n is an arbitrary integer.

In the example shown in FIG. 3, when the width of the contour line is 2w, the value of the pixel having the Z coordinate value satisfying np−w <Z <np + w is set as the value (luminance value) indicating the contour line 104. The luminance after contour composition of the pixel corresponding to the contour line is a value clearly lower than the luminance in the texture image of the same pixel (for example, the luminance is lower than the luminance of the texture image by a predetermined value, or the luminance indicating black with a density of 100%). ). This example is the same as the contour image of the form shown in FIG.

In the example shown in FIG. 4, the contour image is an image in which light and dark are switched for each contour pitch p. In the composite image 110B of this example, for a pixel having a Z coordinate value satisfying 2np <Z ≦ (2n + 1) p, the brightness value of the pixel is lower than the brightness value of the texture image, and (2n + 1) p <Z. For a pixel having a Z coordinate value satisfying ≦ (2n + 2) p, the brightness value of the pixel is higher than the brightness value of the texture image. Here, the brightness difference between the composite image 110B and the texture image may be a fixed value or may be variable according to the brightness value of the texture image (for example, the higher the brightness value of the texture image, the larger the brightness difference). May be good.

In the example shown in FIG. 5, the brightness (density) of the contour image gradually changes according to the Z coordinate within the width of the contour pitch p. In the composite image 110C of the illustrated example, the luminance value of the pixel having the Z coordinate value satisfying 2np <Z ≦ (2n + 1) p is a value obtained by linearly increasing the luminance value of the pixel of the texture image according to the Z coordinate value. Therefore, the luminance value of the pixel having the Z coordinate value satisfying (2n + 1) p <Z ≦ (2n + 2) p is a value obtained by linearly reducing the luminance value of the pixel of the texture image according to the Z coordinate value. The amount of increase or decrease according to the Z coordinate value is set to be 0 when Z = 2np + p / 2.

In the example shown in FIG. 6, the contour image has different colors periodically for each contour pitch p. In this example, the height Z is divided into a plurality of ranges by dividing each contour pitch p. That is, each range satisfies np <Z ≦ (n + 1) p. Then, a specific color is assigned in advance for each value of n, and the color of the pixel of the texture image is corrected with a specific color corresponding to the Z coordinate value of the pixel. The color for each n value of the contour image is specified by, for example, a fixed RGB value. Alternatively, as another example, only the hue may be specified as the color for each n value of the contour image, and the brightness may be changed according to the brightness of the texture image. The same color may be used a plurality of times as the color for each value of n. Further, it is preferable from the viewpoint of visibility that colors having hues as far as possible are assigned to adjacent pitches.

In any of the examples described above, it is possible to arbitrarily adjust the drawing position of the contour lines by arbitrarily translating the 3D shape data in the Z direction. Further, the pitch value may be specified by the user. All of the methods described above show an example of modulating the luminance value of the contour image as a periodic function of the Z coordinate value. The periodic function used for the modulation is not limited to the one illustrated above, and other examples such as a sine wave, a trapezoidal wave, and a staircase wave may be used.

<Embodiment 2>
Next, another embodiment of the present invention will be described.

As mentioned above, the contour image (that is, the moire pattern) of the image taken by the moire camera may differ greatly due to the slight difference in the posture of the evaluation target person, and if the image cannot be taken in the proper posture, the image is taken. It needs to be redone.

In order to deal with this problem, in this embodiment, by generating contour images for several different observation directions from the 3D shape data, a plurality of images viewed from a plurality of observation directions are virtually taken in one shot. Generates a diagnostic image (ie, a composite image) of.

FIG. 7 illustrates the functional configuration of the back-diagnosis image generation system of this embodiment. In FIG. 7, the same elements as those shown in FIG. 1 are designated by the same reference numerals, and the description thereof will be omitted.

In this example, the contour image generation unit 14A generates contour images when viewed from a plurality of different observation directions with respect to the input 3D shape data on the back surface. For example, the contour line image generation unit 14A performs coordinate conversion of 3D shape data so that the observation direction is the Z-axis direction for each of a plurality of different observation directions determined in advance, and the contour line image is converted from the 3D shape data after the coordinate conversion. To generate.

In the system of the first embodiment illustrated in FIG. 1, as shown in FIG. 8, contour line images were generated with the front direction as the Z direction from the viewpoint position of the actual scanner (that is, the 3D measuring device 10). In FIG. 8, the curve 200 indicates the back surface of the evaluation target person, and the arrow indicates the observation direction (Z direction). Further, the straight lines at equal intervals intersecting the curve 200 showing the back surface indicate the height of each contour line pitch p.

The contour image generation unit 14A of the present embodiment includes not only the contour image when viewed from the actual scanner position in the front direction, but also the contour image when the 3D shape data is viewed from another observation direction as shown in FIG. Generate. For example, a plurality of observation directions having different angles, for example, in increments of 1 degree, are set in advance in both positive and negative directions with respect to the front direction of the actual scanner, and contour line images for each of these observation directions are generated. The variation of the observation direction is not limited to the direction of different angles in the horizontal plane, and may include three-dimensionally different directions such as vertical and diagonal.

The image synthesizing unit 16A synthesizes the contour line images generated by the contour line image generation unit 14A for a plurality of different observation directions with the texture image of the back surface of the evaluation target person input from the data separation unit 12, respectively. As a result, a plurality of composite images having different observation directions of contour lines are generated.

The output unit 18A outputs a plurality of generated composite images having different observation directions. For example, in the case of screen display output, the plurality of composite images are displayed in a list or individually. In the case of list display, a composite image that combines contour images of the front direction of the actual scanner is placed in the center of the screen, and the composite image that corresponds to the observation direction closer to the front direction is closer to the center of the screen. Adopt an arrangement that makes it easy to intuitively understand from which direction each composite image shows the contour lines, such as the arrangement.

FIG. 10 illustrates a display screen of a composite image generated by the output unit 18A. The illustrated screen 300 displays five composite images 302 to 310 of the person currently being evaluated. These five composite images 302 to 310 are five different observations set in a horizontal plane including the front direction of the actual scanner (that is, the shooting direction of the 3D measuring device 10, for example, the optical axis direction of the camera used for the shooting). It is a composite image about the direction. The center of the five is the composite image 302 currently of interest, which is displayed in a larger size than the other four. Of the five composite images 302 to 310 displayed on the screen 300, the image displayed in the largest size in the center is called the attention image. On both sides of the attention image (that is, the composite image 302 at this point), among the plurality of observation directions set in the horizontal plane, the observation directions on both sides of the observation direction of the attention image (that is, the angle difference from the observation direction of the attention image). The composite images 304 and 308 for the two observation directions on the left and right with the smallest size are displayed in a size smaller than that of the image of interest. On the outside of these two composite images 304 and 308, composite images 306 and 310 for the observation direction one more adjacent to each other on the left and right are displayed in a smaller size. By selecting the attention image by, for example, a click operation, the attention image may be displayed in a larger size.

When the user "presses" the scroll button 312 on the left side by a touch operation, a click operation, or the like, the output unit 18A changes the image of interest to the image on the left side. That is, the images displayed on the screen 300 are moved to the right one by one. As a result, the composite image 304, which was one to the left of 302 of the composite image in the center, moves to the center and becomes the attention image, and the other images also move to the right position one by one, corresponding to each position. It is displayed in the size you want. If the composite image 310 at the right end before this movement is out of the five display targets and is no longer displayed, and instead there is a composite image for the observation direction to the left of the composite image 306 at the left end. , This composite image is newly displayed at the leftmost position. On the contrary, when the user presses the scroll button 314 on the right side, the output unit 18A changes the image of interest to the image on the right one, and moves the other images to the left one by one accordingly.

The user operates the scroll buttons 312 and 314 to check each composite image while switching the attention image displayed in the center, and the feature of the back surface (spine, etc.) of the evaluation target person (patient, etc.) is the most. Select the composite image that appears well as the final image of interest. For example, in the diagnosis of scoliosis, the left-right asymmetry of the contour stripe pattern is evaluated, but the left-right asymmetry of the contour stripe pattern increases as the observation direction deviates from the appropriate direction. Therefore, the user scrolls the image displayed on the screen 300 left and right, and selects the composite image having as little left-right asymmetry as possible as the final attention image among the plurality of composite images. Then, when the user presses the "Save and Next" button 316 by a touch operation or the like, the output unit 18A displays the image of interest at that time (that is, the largest image displayed in the center) as the current evaluation target. Save it in the diagnostic information database as a human diagnostic image. If a suitable image is not found even after looking at all the composite images generated this time for that person, the user presses the "retake" button 318. As a result, the 3D measuring device 10 is ready for shooting, and another shooting is performed by a separate user operation. When the user presses the "return to initial screen" button 320, the screen 300 returns to the initial state in which the composite image in the front direction of the 3D measuring device 10 is displayed in the center as the image of interest.

Although not shown in FIG. 10, information indicating the observation direction of the composite image 302 may be displayed in the vicinity of the composite image 302 in the center. The information indicating this observation direction is, for example, "0 (front)" (front direction of the actual scanner), "+1" (observation direction one right from the front direction), and "+2" (observation two right from the front direction). Direction), "-1" (observation direction one left from the front direction), "-2", etc. may be used. Not only the composite image 302 in the center, but also all the composite images 302 to 310 displayed on the screen 300 may be displayed with information indicating the observation direction of the images.

In addition, the name, ID, and the like of the evaluation target person displayed on the screen may be further displayed on the screen 300.

Since it takes some time to generate contour images from the 3D measurement data on the back of the evaluation target, wait for the composite images for the five observation directions to be completed, and then display the five composite images on the screen 300. If it is displayed, it may make the user feel that the waiting time is long. Therefore, when the composite image of the front direction of the actual scanner is completed, it is first displayed in the center of the screen 300, then the composite image is generated in order from the observation direction closer to the front direction, and displayed on the screen 300 as soon as it is generated. You may. In addition, the device generates a composite image to be newly displayed when the user presses the scroll button 314 in the background, and when the scroll button is actually pressed, the corresponding composite image is generated. It may be possible to display it quickly.

<Embodiment 3>
This embodiment is an improved version of the second embodiment described above.

In the second embodiment, the image synthesizing unit 16A synthesizes the contour line images generated by the contour line image generation unit 14A for each observation direction with the same texture image output from the data separation unit 12. In this method, when the texture image is combined with the contour image in the observation direction different from the observation direction (that is, the front direction of the actual scanner), the composite image may be slightly uncomfortable.

Therefore, in the third embodiment, the texture image is converted into an image showing a state viewed from the same observation direction as the contour image, and the texture image after this conversion is combined with the contour image.

FIG. 11 illustrates the functional configuration of the back-side diagnostic image generation system of the third embodiment. In FIG. 11, elements similar to the elements shown in FIGS. 1 or 7 are designated by the same reference numerals, and description thereof will be omitted.

The system shown in FIG. 11 is a system in which the multi-directional processing unit 22 is added to the system shown in FIG. The multi-directional processing unit 22 is a texture image (that is, a virtual image described later) in a state of being viewed from the same observation direction as the observation direction of each contour image generated by the contour image generation unit 14A from the texture image input from the data separation unit 12. Texture image) is generated. The texture image viewed from each observation direction is generated, for example, by applying a geometric transformation to the original texture image (the one viewed from the front of the actual scanner). As the geometric transformation, for example, an affine transformation may be used.

A specific example of the texture image generation process corresponding to each observation direction executed by the multidirectional processing unit 22 will be described with reference to FIG.

In this process, the virtual scanner position is determined in the observation direction in which the texture image is generated. For example, the origin is set on the line in the front direction of the real scanner, and the position in the observation direction from the origin at the same distance as the distance between the real scanner and the origin is defined as the virtual scanner position. It is assumed that a virtual 3D measuring device 10 (called a virtual scanner) is arranged at the virtual scanner position. The observation direction is the front direction of the virtual scanner.

When the position and orientation of the virtual scanner are determined in this way, the i-th (i is a positive integer) 3D coordinate point V _i on the 3D shape data and the corresponding 2D coordinate point P on the image plane of the virtual scanner. ^' The relationship with _i is decided. Further, the relationship between the 3D coordinate point V _i and the 2D coordinate point P _i , which is a corresponding point on the actual texture image viewed from the position of the actual scanner, is a texture mapping relationship and is known. That 2D coordinate point P on the image plane of the 2D coordinate point P _i and the virtual scanner on the real texture image _^'i represents the same _{point, P i = (x i,} y i) P ^from' _i = ( x ^{_'i, y'} when the deformation to _i) is expressed by the affine transformation
Can be expressed as. When equation (1) is solved by the least squares method as simultaneous equations for all 2D coordinate points P in which the corresponding 3D coordinate points exist, the affine transformation parameters a _{11 to} a ₃₃ are obtained. By transforming the entire real texture image by this affine transformation parameter, a virtual texture image showing a state in which the back surface of the evaluation target is viewed from the virtual scanner in the observation direction can be obtained. Instead of applying affine transformation to all points P on the real texture image, the luminance value at the point P _i on the corresponding real texture image for point P _^'i on the virtual texture corresponding to the 3D coordinate points V _i adopted may be adopted a luminance value of Miko real texture image deformed using the affine transformation parameters for the point P ^'on the virtual texture 3D coordinate points V _i does not exist.

By such deformation processing, a virtual texture image when viewed from an observation direction different from the front direction of the actual scanner is generated from the actual texture image. The image in the trapezoidal deformed state shown on the right side of FIG. 12 is the virtual texture image 100A. By synthesizing the contour image 102A in the same observation direction with this, a composite image 110A in which the texture image is also viewed from the observation direction can be obtained.

The multi-directional processing unit 22 inputs the generated virtual texture image for each observation direction to the image synthesis unit 16A in association with, for example, the direction ID which is the identification information of the observation direction. Similarly, the contour line image generation unit 14A inputs the generated contour line image for each observation direction to the image synthesis unit 16A in association with, for example, the direction ID.

The image synthesizing unit 16A synthesizes a texture image and a contour image for each observing direction to generate a composite image in a state viewed from the observing direction. The association between the texture image to be combined and the contour image at this time may be performed by using, for example, the above-mentioned direction ID.

In the second and third embodiments, an example in which a composite image for a plurality of observation directions is generated by one photographing of the 3D measuring device 10 has been described, but the methods of the second and third embodiments have a “plurality” of observation directions. Of course, it is applicable not only to the case of generating the composite image of the above, but also to the case of generating the composite image in one direction different from the shooting direction.

<Embodiment 4>
Yet another embodiment will be described with reference to FIG. The system of this embodiment shown in FIG. 13 is obtained by adding a background processing unit 24 to the system shown in FIG.

In this example, the data separation unit 12 inputs both the 3D shape data obtained from the 3D measuring device 10 and the texture image to the background processing unit 24.

The background processing unit 24 classifies each 3D coordinate point indicated by the 3D shape data into one belonging to the back surface of the evaluation target person and one belonging to the background. In this classification, for example, a threshold value is set for the Z coordinate of 3D shape data (that is, distance information along the front direction of the 3D measuring device 10), and whether the Z coordinate of each 3D coordinate point is larger or smaller than the threshold value. , Determine whether the coordinate point is on the back or background of the evaluation target person. As the threshold value, a distance capable of distinguishing the evaluation target person from the background (for example, a wall surface) may be set.

Then, the background processing unit 24 specifies a 2D point cloud on the texture image corresponding to the 3D coordinate point cloud classified as the background, and sets the pixel value (for example, a color value such as RGB) of the specified 2D point cloud as the background. Change to the specific value shown. As a result, the background of the evaluation target becomes a single color. The specific value indicating the background is, for example, a value indicating a color that can be clearly distinguished from the color of the back of the evaluation subject.

The image composition unit 16 generates a composite image by synthesizing the texture image whose background is a single color by the background processing unit 24 and the contour image from the contour image generation unit 14.

According to this embodiment, since the background pattern is removed, the body shape of the evaluation target person stands out in the composite image. As a result, the interpretation of the composite image including the contour lines is performed without being affected by the background pattern. In particular, since the contour of the back of the evaluation subject is less likely to be obstructed by the background pattern, the left-right asymmetry of the contour becomes easier to understand, and the accuracy of interpretation is expected to be improved.

Note that the contour line image generation unit 14 may not generate contour lines in the region determined to be the background, as in the background processing unit 24.

The background removal process in the fourth embodiment can be combined with the methods of the second and third embodiments.

The embodiments of the present invention have been described above. The system of each embodiment described above is realized, for example, by causing a computer to execute a program describing the functions of each element of the system described above. Here, as hardware, for example, the computer includes a microprocessor such as a CPU, a memory (primary storage) such as a random access memory (RAM) and a read-only memory (ROM), a flash memory, an SSD (solid state drive), and an HDD. A controller that controls a fixed storage device such as a (hardware drive), various I / O (input / output) interfaces, a network interface that controls connection to a network such as a local area network, etc., are provided via, for example, a bus. It has a connected circuit configuration. A program in which the processing contents of each of these functions are described is saved in a fixed storage device such as a flash memory via a network or the like, and is installed in a computer. By reading the program stored in the fixed storage device into the RAM and executing it by a microprocessor such as a CPU, the functional module group illustrated above is realized.

Further, each element of each system shown in FIGS. 1, 7, 11, 13, 13 and the like may be distributed and arranged in a plurality of computers connected via a network. For example, on the site on the user (for example, doctor) side, a 3D measuring device that three-dimensionally measures the back surface of the evaluation target person and a computer that functions as an output (for example, screen display) of the output units 18 and 18A are arranged. It is conceivable that one or more computers that perform the functions of the data separation unit 12, the contour image generation units 14, 14A, and the image composition units 16 and 16A are provided on the server side configured as a cloud system or the like.

10 3D measuring device, 12 data separation unit, 14, 14A contour image generation unit, 16, 16A image composition unit, 18, 18A output unit, 22 multi-directional processing unit, 24 background processing unit.

Claims (10)

A means for acquiring the texture image and three-dimensional shape data of the back surface obtained by photographing the back surface of the evaluation target person, and
A generation means for generating a contour image representing the contour line of the back surface from the three-dimensional shape data, and
A compositing means for generating a composite image by compositing the contour image generated by the generating means and the texture image, and
An output means that outputs a composite image generated by the composite means, and
Only including,
The generation means generates contour images of the back surface for each of a plurality of different observation directions from the three-dimensional shape data.
The synthesizing means generates a composite image obtained by synthesizing the contour image and the texture image for the observation direction for each of the plurality of observation directions.
Image generator. For each of the plurality of observation directions, a virtual texture generation means for generating a texture image representing the back surface when virtually viewed from the observation direction is further included.
The synthesizing means generates a composite image obtained by synthesizing the contour image and the texture image for the observation direction for each of the plurality of observation directions.
The image generator according to claim 1 . The output means
A means for displaying the composite image for each of the plurality of observation directions generated by the synthesis means on the screen, and
A means for accepting selection from the user among the composite images for each of these observation directions,
A means of associating the composite image selected from the user with the identification information of the evaluation target person and registering the composite image in a predetermined database.
The image generator according to claim 1 or 2 . A means for acquiring the texture image and three-dimensional shape data of the back surface obtained by photographing the back surface of the evaluation target person, and
A generation means for generating a contour image representing the contour line of the back surface from the three-dimensional shape data, and
A compositing means for generating a composite image by compositing the contour image generated by the generating means and the texture image, and
An output means that outputs a composite image generated by the composite means, and
Including
The generation means generates a contour image from the three-dimensional shape data when the observation direction is a second direction different from the shooting direction when the back surface of the evaluation target person is photographed.
The synthesizing means synthesizes the contour image and the texture image when the second direction is the observation direction.
Images generator. Further including a virtual texture generation means for generating a texture image when the back surface is virtually viewed from the second direction, which is the observation direction, from the texture image.
The compositing means generates a composite image obtained by compositing the contour image and the texture image in the second direction.
The image generator according to claim 4 . A processing means for identifying a background portion of the texture image that is the background of the back surface based on the three-dimensional shape data, and processing the image of the background portion of the texture image so as to be distinguishable from the back surface. , Including
The synthesizing means synthesizes the texture image processed by the processing means and the contour image to generate the synthesizing image.
The image generator according to any one of claims 1 to 5 . A step of acquiring the texture image and three-dimensional shape data of the back surface obtained by photographing the back surface of the evaluation target person, and
A step of generating a contour image representing the contour line of the back surface from the three-dimensional shape data, and
A step of synthesizing the generated contour image and the texture image to generate a composite image,
Steps to output the generated composite image and
Including
In the step of generating the contour image, the contour image of the back surface for each of the plurality of different observation directions is generated from the three-dimensional shape data.
In the step of generating the composite image, an image generation method for generating a composite image in which the contour image and the texture image in the observation direction are combined for each of the plurality of observation directions . A step of acquiring the texture image and three-dimensional shape data of the back surface obtained by photographing the back surface of the evaluation target person, and
A step of generating a contour image representing the contour line of the back surface from the three-dimensional shape data, and
A step of synthesizing the generated contour image and the texture image to generate a composite image,
Steps to output the generated composite image and
Including
In the step of generating the contour image, the contour image is generated from the three-dimensional shape data when the observation direction is a second direction different from the shooting direction when the back surface of the evaluation target person is photographed.
In the step of generating the composite image, an image generation method of synthesizing the contour image and the texture image when the second direction is the observation direction. On the computer
A step of acquiring the texture image and three-dimensional shape data of the back surface obtained by photographing the back surface of the evaluation target person, and
A step of generating a contour image representing the contour line of the back surface from the three-dimensional shape data, and
A step of synthesizing the generated contour image and the texture image to generate a composite image,
Steps to output the generated composite image and
A program for executing,
In the step of generating the contour image, the contour image of the back surface for each of the plurality of different observation directions is generated from the three-dimensional shape data.
The program for generating the composite image is characterized in that, for each of the plurality of observation directions, a composite image in which the contour image and the texture image in the observation direction are combined is generated . On the computer
A step of acquiring the texture image and three-dimensional shape data of the back surface obtained by photographing the back surface of the evaluation target person, and
A step of generating a contour image representing the contour line of the back surface from the three-dimensional shape data, and
A step of synthesizing the generated contour image and the texture image to generate a composite image,
Steps to output the generated composite image and
It is a program to execute
In the step of generating the contour image, the contour image is generated from the three-dimensional shape data when the observation direction is a second direction different from the shooting direction when the back surface of the evaluation target person is photographed.
A program characterized in that in the step of generating the composite image, the contour image and the texture image when the second direction is set as the observation direction are combined .