JP2022076541A - Method for presenting composition of real-estate living room image - Google Patents

Abstract

To provide a method of presenting the composition of a real-estate living room image to support photographing of a real-estate living room image constructed by only a composition.SOLUTION: In a method for presenting the composition of a real-estate living room image to present a composition for a camera when an image of a real-estate living room is captured, edge detection 42 is performed on a living room image, straight line detection 43 is performed on the living room image which has been subjected to the edge detection 42, and detection 44 of a composition feature point 11 is performed from a line segment 51 obtained by the straight line detection 43. Further, the living room image is classified into four quadrants by using an X-axis and a Y-axis, and a corner 10 is detected in each of the four quadrants. Furthermore, a composition feature line 20 is estimated based on the detected composition feature point 11 and an extended line segment 51 passing through the corner 10.SELECTED DRAWING: Figure 5

Description

There is an application for application of Article 30, Paragraph 2 of the Patent Act. (2) Reiwa Presented at the lecture of the Winter Games 2019 of the Society of Image Information and Television Engineers on December 13, 1991. (3) On February 26, 2nd year of Reiwa, the details of the lecture paper were announced at the application system of the Study Group of the Imaging Media Society.

The present invention relates to a method for presenting a composition of a real estate living room image for presenting a composition to a camera when the real estate living room image is taken.

In recent years, an increasing number of home buyers are searching for real estate properties and information using the Internet. At that time, many people cite real estate room images as a point for selecting a real estate company, and the appearance of the images is regarded as important. However, since such living room images are taken by employees of a real estate company who are not specialists in photography, the quality and quality of the images vary. Therefore, there is a demand for stable quality living room images with less variation.

On the other hand, Patent Document 1 discloses a technique relating to a photographing device for guiding a photographer's photographing operation to match subject information and photographing composition information in order to support image photographing.

Japanese Unexamined Patent Publication No. 2011-097194

Unlike a general photograph, a real estate living room image does not have a subject and is composed only of a composition (or the entire composition is the subject). The above-mentioned Patent Document 1 describes a technique for adjusting the relationship between the subject and the composition, but does not disclose a method of performing shooting support with only the composition.

An object of the present invention is to provide a method for presenting a composition of a real estate living room image for supporting shooting of a real estate living room image composed only of a composition.

In order to solve the above problems, the present invention is a method for presenting a composition of a real estate living room image for presenting a composition to a camera when an image of a real estate living room is taken, and edge detection is performed on the living room image. It is characterized in that straight line detection is performed on the living room image for which edge detection has been performed, and composition feature points are detected from the line segments obtained by the straight line detection.

Further, the living room image may be classified into four quadrants using the X-axis and the Y-axis, and corners may be detected in each of the four quadrants.

Further, the composition feature line may be estimated based on the detected extended line segment passing through the composition feature point and the corner.

Further, a three-divided composition or a four-divided composition can be selected based on the composition feature points and the composition feature lines.

Furthermore, the aspect ratio of the composition feature line estimated may be used to correct the selected three-division composition or the four-division composition.

According to the present invention, it is possible to select and apply the optimum composition for a living room image based on the detected composition feature points. Therefore, since the balance of the living room image can be confirmed mechanically instead of the sense of the photographer by this composition, it is possible to provide a living room image with stable quality with little variation in image quality and quality.

It is a living room image created by using the composition presenting method of the real estate living room image which concerns on embodiment of this invention, (A) is 1 vanishing point image, (B) is 2 vanishing point image. It is a presentation composition of a vanishing point image, (A) shows a three-division composition, and (B) shows a four-division composition. It is a presentation composition of two vanishing point images, in which (A) is a three-divided composition and composition feature points are arranged at a ratio of 3: 1, (B) is 1: 1 and (C) is 1: 3. .. It is a presentation composition of two vanishing point images, in which (A) is a four-divided composition and composition feature points are arranged at a ratio of 3: 1, (B) is 1: 1 and (C) is 1: 3. .. It is a flowchart which shows the processing outline. It is a schematic diagram which shows the example of the edge detection image. 1 The outline of the composition feature point detection of the vanishing point image is shown, (A) is a line segment detection result, and (B) is a composition feature point detection result. It is a schematic diagram which shows the composition characteristic point classified into four quadrants. It is a schematic diagram which shows the estimation result of a composition feature line. The aspect ratio correction results in the three-divided composition, (A) shows no correction, (B) shows horizontal correction, and (C) shows vertical correction. The aspect ratio correction results in the 4-division composition, (A) shows no correction, (B) shows horizontal correction, and (C) shows vertical correction. 2 Vanishing point images, (A) is a line segment detection result, and (B) is a composition feature point detection result. Other two vanishing point images different from those in FIG. 12, (A) is a line segment detection result, and (B) is a composition feature point detection result. 2 It is a schematic diagram which shows composition selection of a vanishing point image, (A) shows composition feature point position, (B) shows division composition selection.

Hereinafter, a method for presenting a composition of a real estate living room image according to an embodiment of the present invention will be described.

As shown in FIGS. 1 (A) and 1 (B), the composition features in the present specification refer to the composition feature points 11 and the composition feature lines 20 appearing on the living room image. Further, the composition feature point 11 refers to a vanishing point in the perspective method, and there may be a plurality of vanishing points depending on the type of the perspective method. In the present invention, an image of 1 vanishing point or 2 vanishing points (hereinafter referred to as 1 vanishing point image and 2 vanishing point images) is targeted, and in the present specification, a living room image is referred to as a 1 vanishing point image or 2 vanishing point images. It shall refer to an image.

1 Vanishing point image 1 is an image in which three surfaces of the front surface and the left and right walls are captured, as shown in FIG. 1 (A). Further, the composition feature line 20 is a boundary line between the floor and the three walls, a boundary line between the ceiling and the three walls, and a boundary line between the three walls and the wall.

2 Vanishing point image 2 is an image showing two walls, a floor, and a ceiling as shown in FIG. 1 (B). Further, as shown in FIG. 1B, the composition feature line 20 in the 2 vanishing point image 2 is a boundary line between two walls, a boundary line between two walls and a ceiling, and a boundary line between two walls and a floor. It becomes a line.

2 to 4 are schematic views of the three-divided composition 31 and the four-divided composition 32.

As the composition of the 1 vanishing point image 1, two types of composition, a three-division composition 31 and a four-division composition 32, which are generally used as a shooting composition, are prepared. Also, when creating a perspective image of a living room in architecture (an image taken by applying a certain composition), the vanishing point is located on the eyeline, so the camera should be at the height of the line of sight when shooting. Assuming that the image is held, the position of the composition feature point 11 is located at the center of the image.

2 Vanishing points The positions of the two composition feature points 11 in the image 2 are the distances from the center of the image as shown in FIGS. 3 (A) to 3 (C) and FIGS. 4 (A) to 4 (C). Prepare three types so that the ratio of is 3: 1, 1: 1, or 1: 3. This is determined so that the position of the corner 10 is a right-angled isosceles triangle and a right-angled triangle of 30 degrees, 60 degrees, and 90 degrees with reference to the equiangular projection method and the dichotomous angle projection method in the perspective image. Because. Further, the position in the vertical direction is set to the center of the image as in the case of the 1 vanishing point image 1.

Further, as shown in FIGS. 3 and 4, two types of composition feature lines (lines drawn vertically and horizontally in FIGS. 2 to 4) are used, that is, a three-division composition 31 and a four-division composition 32. .. A total of 6 types of composition are prepared by combining 2 types of divided composition and 3 types of positions of composition feature points.

FIG. 5 is a flowchart showing a processing outline of the 1 vanishing point image 1 or the 2 vanishing point image 2.

The living room of a real estate property is characterized by parallel walls facing each other, and the floor and ceiling are parallel. These boundary lines can be the composition feature line 20. Further, by using a boundary line on the floor, ceiling or wall, the composition feature point 11 can be detected. Therefore, the edge detection 42 is performed in order to detect the boundary line described above, but when the edge detection 42 is difficult because the colors of the wall and the floor are similar in the living room image and there are many white walls. There is. Therefore, the edge detection 42 is performed after the contrast adjustment 41 of the image is performed as the preprocessing A.

As a method of contrast adjustment 41, CLAHE (Contrast Limited Adaptive Histogram Equalization) is used. CLAHE is an adaptive histogram flattening with contrast limits. The image is divided into small areas, and the histogram is flattened for each area. However. If there are extremely bright pixels in a small area, the contrast cannot be adjusted properly. Therefore, an upper limit value is set in the histogram, and the histogram exceeding the upper limit value is evenly distributed in a small area, and then the histogram flattening is performed. By using CLAHE, it is possible to adjust the contrast by emphasizing the edges even when the light source is reflected in the image or even in a room image where the outside light is strong. In the present invention, the image is divided into 8 × 8 pixels, and the upper limit of the contrast is set to 40.

The edge detection 42 is performed by using the canny method for the image for which the contrast has been adjusted. First, the image is smoothed by applying a 5 × 5 pixel Gaussian filter. Next, the RGB color image is decomposed into R, G, and B components, and edge detection 42 is performed for each component using the canny method. Then, as shown in FIG. 6, a binary image 48 (edge detection image) is generated using the maximum value of the output value of each component.

Next, as the composition feature detection step B, the straight line detection 43 is performed using the Hough transform, and the composition feature point detection 44 is performed from the obtained straight line. Further, the composition feature line is estimated 45 using the composition feature point 11, a composition suitable for either the three-division composition 31 or the four-division composition 32 described above is selected, and the presentation is performed after the correction 46 is performed.

The straight line detection 43, the composition feature point detection 44, and the composition feature line estimation 45 detect line segments and straight lines using the Hough transform on the binary image 48 obtained by the edge detection 42. In FIG. 7A, the line segment detected by the Hough transform is represented by 51, and the end points of the line segment are represented by round points 52. A threshold value is set for the angle of the detected straight line, and the vertical / horizontal and linear components parallel to the optical axis of the camera are detected.

Further, in FIG. 7B, the vertical component and the horizontal component are the two-dot chain line 53, and among the parallel components, the straight line having the inclination θ of 20 degrees <θ <70 degrees is the solid line 54, and the straight line of 110 degrees <θ <160 degrees is drawn. It is represented by a dotted line 55. Further, the average value of the coordinates of the intersection of the straight lines of the dotted line 55 and the solid line 54 is detected as the composition feature point 11. The black circles represent the detected composition feature points 11.

The estimation 45 of the composition feature line 20 is performed using the two endpoints 52 of the line segment 51 obtained by the Hough transform. As shown in FIG. 8, classification is performed into four quadrants with the corner 10 as the origin. Then, the endpoint 52 is selected from each of the four quadrants, and the four corner candidates 52a that can form a rectangle that is the boundary of the front wall are selected.

First, one end point 52a in the second quadrant is selected. A point having the same Y coordinate as the selected endpoint 52a is searched from the first quadrant. When the end point 52a exists, the point having the same X coordinate as the first selected point is searched from the third quadrant. If the end point 52a does not exist (is not detected), the end point 52a next to the second quadrant is selected, and the end point of the first quadrant is searched in the same manner. When the end point 52a of the third quadrant exists, the end point 52a having the X coordinate of the end point of the first quadrant and the Y coordinate of the end point of the third quadrant is searched in the fourth quadrant.

When the end point 52a exists, the boundary line between the wall and the ceiling and the wall and the floor is estimated using these four points. If the searched endpoint 52 does not exist in the third or fourth quadrant, the next endpoint 52 in the second quadrant is selected. At this time, the coordinates of the end points to be searched have a width of ± 3pixel and are searched. After taking out the four corner candidates 52a forming the rectangle, a straight line is extended from the composition feature point 11 to the four corner candidates 52a. The number of end points 52 on the straight line is counted as the score of the four corner candidates 52a. At this time, it is assumed that the coordinates of the four corner candidates 52a are (Cx, Cy), the composition feature points are (VPx, VPy), and the end points are (x1, y1), and the end points 52a satisfying the following equation are on a straight line. ..

Scores are counted for all four corner candidates 52a that can form a rectangle, and the line segment connecting the ones with the highest scores is defined as the composition feature line 20. FIG. 9 shows the estimation result of the composition feature line 20.

Based on the estimated composition feature line 20, an appropriate composition is determined and presented from the prepared compositions. At that time, since the aspect ratio of the wall differs depending on the living room, even if the divided composition is presented as it is, it does not match the composition feature line 20. Therefore, the composition correction 46 prepared by using the aspect ratio of the composition feature line 20 is performed. Aspects as shown in FIGS. 10 (a) to 10 (c) and FIGS. 11 (a) to 11 (c) based on the composition feature line 20 in which the dividing lines of the three-divided composition 31 and the four-divided composition 32 are detected. Correct the ratio vertically and horizontally.

The distance between each side of the rectangle of the obtained composition feature line 20 and the dividing line of each composition is calculated, and the composition having the shortest total distance is selected and presented. In the case of this image, a quadrant composition corrected in the vertical direction is selected.

The above-mentioned explanation using FIGS. 7 to 11 shows the composition presentation method in the 1 vanishing point image 1, but the same can be applied to the 2 vanishing point image 2. Hereinafter, the case of the 2 vanishing point image 2 will be described with reference to FIGS. 12 and 13.

1 Vanishing point Similar to the image 1, line segments and straight lines are detected from the edge image using the Hough transform. In FIG. 12A, the line segment detected by the Hough transform is represented by a solid line 51, and the end point of the line segment is represented by a round point 52. The detected straight line is classified by setting a threshold value for the angle, and the horizontal component and the straight line converging to the two composition feature points 11 are detected.

In FIG. 12B, the vertical component is represented by a two-dot chain line 61, a straight line that converges to the composition feature point 11 on the left is represented by a dotted line 63, and a straight line that converges to the composition feature point 11 on the right is represented by a solid line 62. The two composition feature points 11 detect each of the intersection of the dotted line 63 and the intersection of the solid line 62 using an average value. At this time, the intersections that are noise with a distance of 80 pixels or more from the detected composition feature points 11 are excluded, and the average value of the remaining intersections is used for re-detection. This is repeated until there are no exclusion points, and the composition feature point 11 is determined.

The composition feature line 20 is estimated using the end points 52 (round points) of the solid line 51. The detected end points 52 are classified into those having a larger Y coordinate than the composition feature point 11 and those having a smaller Y coordinate. After that, the upper end points are selected in order, and the end points 52 having the same X coordinate are searched from the lower end points 52. When the lower end point 52 is present, the composition feature line 20 is estimated using the two end points 52. A straight line is extended from the composition feature point 11 to the two end points 52, and the number of end points 52 on the straight line is counted as a score as in the case of the one vanishing point image 1.

If the lower end point 52 does not exist, the upper point is selected again and the lower end point 52 is searched. However, in the case of the 2 vanishing point image 2, since the corner 10 of the living room is in the back, as shown in the Z portion of FIG. 13 (a), the line segment may not be detected up to the corner 10 due to the influence of a shadow or the like. be. Therefore, the point of the corner candidate 52a is added from the detected straight line. In FIG. 13B, the intersection of the dotted line 63 and the solid line 62 is represented by a square point 64. By adding the four corner points 64 as the points of the corner candidate 52a, the corner 10 in the back of the room can be detected.

The score is counted at all two points having the same X coordinate, and the one with the highest score is defined as the composition feature line 20. In addition, the line segment connecting the two points is defined as the boundary line between the walls.

Based on the estimated composition feature line 20, an appropriate composition is determined and presented from the prepared composition. First, the position of the composition feature point 11 is determined. The distance between the two detected composition feature points 11 is measured, and the distance obtained by dividing the distance into four equal parts is defined as d. As shown in FIG. 14A, points 11a, 11b, and 11c are prepared in which the ratio of the distances from the positions of the composition feature points 11 to the center is 1: 3, 1: 1, 3: 1. The distance between each of the points 11a, 11b, 11c and the composition feature point 11 is measured, and the closest one is selected.

Next, the division composition is determined. Among the detected composition feature lines 20, the boundary line between walls is used. As shown in FIG. 14B, the dividing line 31a of the three-divided composition 31 and the dividing line 32a of the four-divided composition 32 are prepared, respectively. The distance between the wall-to-wall boundary line and the dividing lines 31a and 32a is measured, and the nearest dividing line 31a and 31b are selected. After selecting the presentation composition, correction is performed using the wall-to-wall boundary line of the composition feature line 20 in the same manner as in the determination of the dividing lines 31a and 31b. This correction measures the distance of the composition feature line 20 and determines the composition presentation position at the same distance from the center.

By the above method, the composition presentation was verified for each of the 1 vanishing point image 1 and the 2 vanishing point image 2. In the 1 vanishing point image 1, the composition could be presented to 5 of the 9 images. Further, in the 2 vanishing point image 2, the composition could be presented to 22 of the 27 images.

The method for presenting the composition of a real estate living room image according to the present invention can be used by installing (storing) it as an application on a camera-equipped mobile phone or smartphone (hereinafter referred to as a mobile terminal) used by an employee of a real estate company who is a photographer. can. For example, the method of the present invention is used to present a composition on the screen of a mobile terminal to support the shooting operation of the photographer. This shooting support, for example, compares the image on the screen with the composition and instructs the photographer to move the camera in either the up, down, left, or right direction, or the living room image displayed on the screen. It suggests that the balance is good or bad. Instructions or suggestions for such operations can be displayed in characters or symbols on the screen of the mobile terminal, or can be transmitted to the photographer by voice.

It is also possible to provide an automatic shooting mode in which the photographer automatically shoots an image (captures it as data) when the image on the screen matches the presented composition when shooting with a mobile terminal. can. With this automatic shooting mode, there is no risk of missing a momentary photo opportunity, and photography can be easily performed. Further, by eliminating the trouble of manually operating the shutter by the photographer, it is possible to reduce the shooting operation of the photographer. Further, a mode switching function capable of selecting an automatic shooting mode and a manual shooting mode may be provided so that the shutter can be manually operated as needed.

In addition, a personal computer that functions as a server and a camera that is connected to this server via an internet line will be used, and the composition will be presented on the screen of this camera according to a command from the personal computer, and shooting support will be provided.

Furthermore, by capturing the image data of the living room taken by a personal computer equipped with an image processing function and presenting the composition to the taken living room image, an image of a well-balanced composition can be seen from a part of the image having an unbalanced shooting as a whole. Can be extracted. In this case, it is possible to add a function of trimming an unnecessary area in the image.

It is also possible to shoot a moving image using a mobile terminal or a camera and present a composition to the moving image data so that a well-balanced one-shot image in the moving image can be extracted as a screenshot. This screenshot can be done automatically or manually. This work may be performed automatically by an application installed in a mobile terminal or a camera, or may be performed by using software on a personal computer that has captured video data.

The personal computer described above includes, for example, a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), an HDD (Hard Disk Drive), a GPU (Graphical Processing Unit), an I / F (Interface), and the like. It has a bus and a display device. An SSD (Solid State Drive) may be provided in place of the HDD or in addition to the HDD. Inside the HDD or SSD, software (program) for executing the method for presenting the composition of the real estate living room image of the present invention is stored, and these hardware resources and software resources cooperate to perform processing. It has become like.

In the method for presenting the composition of the real estate living room image according to the embodiment of the present invention, the edge detection 42 is performed on the living room image by using the canny method, and the Hough transform is used for the living room image on which the edge detection 42 is performed. Since the straight line detection 43 is performed and the composition feature point 11 is detected from the line segment 51 obtained by the straight line detection 43, the optimum composition for the living room image is selected and applied based on the detected composition feature point 11. Can be done. Therefore, since the balance of the living room image can be confirmed mechanically instead of the sense of the photographer by this composition, it is possible to provide a living room image with stable quality with little variation in image quality and quality.

Further, since the living room image is classified into four quadrants using the X-axis and the Y-axis and the corner 10 is detected in each of the four quadrants, the four corners of the wall photographed in a square in front of the image are the corner 10. Easy to detect. Further, the vanishing point can be easily determined from the four corners 10.

Further, since the composition feature line 20 is estimated based on the extended line segment 51 passing through the detected corner 10, even if it is difficult to grasp the boundary lines of the wall, floor, and ceiling from the living room image. , The composition feature line 20 can be estimated.

Further, since the three-division composition or the four-division composition is selected based on the composition feature point 11 and the composition feature line 20, the angles of a plurality of cameras can be selected when taking a living room image, and each of the selected angles can be selected. You can take a well-balanced image.

Furthermore, since the selected 3-division composition or 4-division composition is corrected using the estimated aspect ratio of the composition feature line 20, even if the aspect ratio of the wall differs depending on the living room, it can be adapted to the prepared composition. You can take a well-balanced image.

Although the method for presenting the composition of the real estate living room image according to the embodiment of the present invention has been described above, the present invention is not limited to the above-described embodiment, and various modifications and modifications are made based on the technical idea of the present invention. It can be changed.
In the method of the present invention, the composition characteristics are determined by using the canny method and the Hough transform, but other methods can also be used. Another method is, for example, LSD (Line Segment Detector). The purpose of this LSD is to detect the contour of a straight line locally on an image. A contour is a zone of an image in which the gray level changes from dark to light and vice versa fast enough. As a feature of LSD, there is a tendency to detect a lot of line segments shorter than the Hough transform, so when using LSD, it is necessary to add a step of integrating and processing a plurality of line segments.

Further, even if another method is developed to replace the canny method or the Hough transform in the future, it has the same performance as these methods and can be used as a substitute based on the idea of the present invention. If possible, these other methods shall also be included in the scope of rights.

1 living room image (1 vanishing point image)
2 Living room image (2 vanishing point image)
10 Corner 11 Composition Feature Point 11a Composition Feature Point Position (1: 3)
11b Composition feature point position (1: 1)
11c Composition feature point position (3: 1)
20 Composition feature line 31 3 division composition 31a 3 division line 32 4 division composition 32a 4 division line 41 Contrast adjustment 42 Edge detection 43 Straight line detection 44 Composition feature point detection 45 Composition feature line estimation 46 Correction / selection of shooting composition 48 Binary image 51 Lines 52 End points 52a Corner candidates A Pretreatment B Composition feature detection C Composition presentation

Claims (5)

It is a method of presenting the composition of the real estate living room image to show the composition to the camera when taking the image of the real estate living room.
Edge detection is performed on the living room image, and
A straight line detection was performed on the living room image from which the edge detection was performed.
A method for presenting a composition of a real estate living room image, which comprises detecting a composition feature point from a line segment obtained by the straight line detection. The method for presenting a composition of a real estate living room image according to claim 1, wherein the living room image is classified into four quadrants using the X-axis and the Y-axis, and corners are detected in each of the four quadrants. The method for presenting a composition of a real estate living room image according to claim 2, wherein the composition feature line is estimated based on the detected line segment extending through the composition feature point and the corner. The method for presenting a composition of a real estate living room image according to any one of claims 1 to 3, wherein a three-divided composition or a four-divided composition is selected based on the composition feature points and the composition feature lines. .. The method for presenting a composition of a real estate living room image according to claim 4, wherein the selected three-divided composition or the four-divided composition is corrected using the estimated aspect ratio of the composition feature line.

Images