KR102125563B1 - Device and mathod for gernerating data of representing structure in the room - Google Patents

Abstract

Outputting a predetermined sound signal at a first position and a second position in the room, and obtaining a first reflected signal and a second reflected signal for the output predetermined sound signal, respectively; Estimating distance information for the room using the obtained first and second reflected signals; Obtaining at least one angle information for the room; And generating a floor plan of a room representing the structure of the room using the estimated distance information and the obtained angle information.

Description

DEVICE AND MATHOD FOR GERNERATING DATA OF REPRESENTING STRUCTURE IN THE ROOM}

The present invention relates to a method for generating data representing the structure of a room and a device for performing the same.

Recently, with the rapid development of the electronics industry and communication technology, navigation services using portable devices are widely used by users. When a user uses the navigation service through a portable device, a navigation service for the interior of the building may not be provided. If there is information indicating the structure of the inside of the building, such as a floor plan of the building, the navigation service for the inside of the building may be provided using this, but if there is no information, the user may not be able to use the navigation service inside the building.

Accordingly, methods of constructing a structure related to the interior of a building using a portable device have been studied. However, since most of the methods use an apparatus having a special performance, a method for immediately constructing a structure related to the interior of a building using a general portable device is required.

It is to provide a method for generating data representing the structure of a room and a device for performing the same. Another object of the present invention is to provide a recording medium readable by a computer on which a program for executing the method is executed on a computer. The technical problems to be achieved by the present embodiment are not limited to the technical problems as described above, and other technical problems may be inferred from the following embodiments.

According to an aspect, a method for generating data indicating a structure of a room in a device includes outputting a predetermined sound signal at a first position and a second position in the room, and a first for the output sound signal. Obtaining a reflection signal and a second reflection signal, respectively; Estimating distance information for the room using the obtained first and second reflected signals; Obtaining at least one angle information for the room; And generating a plan view of the room representing the structure of the room using the estimated distance information and the obtained angle information.

In addition, the method may further include generating a 3D model representing the structure of the room using the generated floor plan.

In addition, the method may further include generating a 3D model representing the structure of the room using the generated floor plan and information on objects located in the room.

Further, the distance information for the room may be information indicating a distance from the device to at least one of the walls of the room or objects located in the room.

In addition, estimating the distance information for the room may be estimating the distance information for the room based on the distance between the first position and the second position.

In addition, estimating distance information for the indoor may include: mapping the first reflected signal and the second reflected signal based on the distance between the first position and the second position; And estimating distance information for the room based on the mapping.

In addition, the mapping of the first reflection signal and the second reflection signal may include using the time required for the predetermined acoustic signal to move a distance between the first position and the second position, to the first reflection signal. And mapping the second reflected signal.

In addition, the first reflection signal and the second reflection signal are acoustic signals where the predetermined acoustic signal is reflected on at least one wall or object in the room and returned to the device, and estimates distance information for the room The step of performing may be estimating information indicating a distance from the device to at least one wall or object in the room.

In addition, the step of acquiring at least one angle information for the room may be to obtain at least one of an angle of at least one wall of the room or an angle of a corner of the walls of the room.

In addition, the obtaining of at least one angle information for the room may include: obtaining at least one angle information for the room based on a user input to at least one wall or at least one corner of the room. ; And storing the obtained at least one angle information.

According to another aspect, the device includes: an audio output unit; An acoustic receiver; A sensing unit for acquiring at least one angle information about the room; And at a first position in the room, control the sound output unit to output a predetermined sound signal, control the sound receiver to receive a first reflected signal for the output predetermined sound signal, and control the indoor And a control unit for controlling the sound output unit to output the predetermined sound signal at 2 positions, and for controlling the sound reception unit to receive a second reflected signal for the outputted sound signal. Estimating distance information for the room using the received first and second reflected signals, and using the estimated distance information and the obtained angle information to obtain a plan view of the room showing the structure of the room It can be characterized by generating.

In addition, the control unit may generate a 3D model showing the structure of the room using the generated floor plan.

In addition, the controller may generate a 3D model representing the structure of the room by using the generated floor plan and information on objects located in the room.

Further, the distance information for the room may be information indicating a distance from the device to at least one of the walls of the room or objects located in the room.

In addition, the controller may estimate distance information for the room based on the distance between the first position and the second position.

In addition, the controller may map the first reflection signal and the second reflection signal based on the distance between the first position and the second position, and estimate distance information for the room based on the mapping. have.

In addition, the controller may map the first reflected signal and the second reflected signal using a time required for the predetermined sound signal to move the distance between the first position and the second position.

In addition, the input unit, the first reflected signal and the second reflected signal that is an acoustic signal that is reflected back to the device by a predetermined acoustic signal output from the output unit is reflected on at least one wall or object in the room Acquiring, and the controller may estimate information indicating a distance from the device to at least one wall or object in the room.

In addition, the sensing unit may acquire at least one of an angle with respect to at least one wall of the room or an angle with respect to a corner where the walls of the room contact.

In addition, the user input unit for acquiring a user's input for at least one wall or at least one corner of the room, the sensing unit, at least one angle to the room based on the input of the acquired user Information can be obtained.

According to various embodiments of the present invention, a device that generates data indicating the structure of a room may immediately identify the structure of the room in which the user is located and provide it to the user. The user can grasp the structure of the room at a glance, and can use the navigation service even indoors. Users can easily grasp the structure of the room anytime, anywhere without special equipment.

In addition, the device may generate a 3D model or a preview model representing the structure of the room and provide it to the user, and the user may 3D print the 3D model or check the preview model in advance as if it is real.

1 is a diagram for describing a method of generating data representing a structure of a room in a device according to an embodiment.
2 shows an example in which data representing a structure of a room is used in a device according to an embodiment.
3 illustrates a hardware configuration of a device generating data representing the structure of a room according to an embodiment.
4 illustrates a process of obtaining reflection signals for a predetermined sound signal in a device according to an embodiment.
5 is a view illustrating a process of post-processing reflection signals obtained from a device according to an embodiment.
6 and 7 illustrate a process of mapping reflected signals obtained from a device according to an embodiment.
8 is a diagram for explaining a process of obtaining angle information about a room in a device according to an embodiment.
9 is a diagram for explaining a process of generating data representing a structure of a room in a device according to an embodiment.
10 illustrates an example of a user interface screen provided to generate data representing a structure of a room in a device according to an embodiment.
11 is a diagram for describing a method of generating data representing a structure of a room in a device according to an embodiment.
12 is a flowchart of a method for generating data representing a structure of a room in a device according to an embodiment.
13 illustrates a hardware configuration of a device that generates data representing the structure of a room according to an embodiment.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the following description, only parts necessary for understanding the operation according to the present invention are described, and descriptions of other parts may be omitted so as not to distract the subject matter of the present invention.

In addition, the terms or words used in the specification and claims to be described below should not be interpreted as being limited to ordinary or dictionary meanings, and meanings that are consistent with the technical spirit of the present invention so that the present invention can be most appropriately expressed. And should be interpreted as a concept.

Throughout the specification, when a part is "connected" to another part, this includes not only "directly connected" but also "electrically connected" with another element in between. . Also, when a part “includes” a certain component, this means that other components may be further included, rather than excluding other components, unless otherwise specified.

Hereinafter, the present invention will be described in detail with reference to the accompanying drawings.

1 is a diagram for describing a method of generating data representing a structure of a room in the device 100 according to an embodiment.

Referring to FIG. 1, a user can grasp the structure of the room 10 using the device 100. Specifically, the device 100 may generate data indicating the structure of the room 10 and provide it to the user, so that the user can grasp the geometry of the room 10 through the device 100.

The device 100 according to an embodiment is a mobile electronic device, a smart phone, a mobile phone, a digital TV, a kiosk, a personal digital assistant (PDA), a laptop, a media player, a global positioning system (GPS) device, a laptop, and others It may be a mobile computing device, but is not limited thereto. In addition, the device 100 may include various devices capable of transmitting and receiving sound signals, such as a speaker and a microphone.

The interior 10 in which the device 100 grasps the structure is an interior of a building in which a user is located, and may be an interior of a house, a shopping center, an office, a factory, or more specifically, an interior of a room. In addition, the interior 10 may include, but is not limited to, a closed or open space including at least one fixed wall.

The data representing the structure of the room 10 may be data visually showing the structure of the device room 10, but is not limited thereto, and may be data in various ways. For example, data representing the structure of the interior 10 may be a plan view 20, a three-dimensional model, a preview model, and the like.

The user can grasp the structure of the room 10 in which the user is located at a glance, and use a navigation service for the structure of the room 10 using data representing the structure of the generated room 10.

The device 100 may output a predetermined sound signal at the first and second positions of the room 10, and obtain a first reflected signal and a second reflected signal for the output predetermined sound signal, respectively. . The user sequentially moves the device 100 to the first position and the second position, so that the device 100 sequentially outputs a predetermined acoustic signal at the first position to obtain the first reflected signal, and the second position to the predetermined position. The second acoustic signal may be obtained by outputting the acoustic signal of.

The device 100 may estimate distance information for the room 10 using the obtained first and second reflected signals. For example, the first reflected signal and the second reflected signal may be reflected signals that are reflected by at least one wall of the room 10 and returned to the device 100 by a predetermined acoustic signal output from the device 100, The distance information for the room 10 may be information indicating a distance from at least one wall of the room 10 reflecting a predetermined acoustic signal to the device 100. Further, the distance information may be information indicating a distance from at least one object located in the room 10 reflecting a predetermined acoustic signal to the device 100. The object located in the room 10 may be any object having a volume that can be placed in the room 10, such as a television, desk, chair, bed, sofa, ornament.

The device 100 may acquire at least one angle information for the room 10. The angle information for the room 10 may be information indicating an angle of at least one wall of the room 10 or an angle of a corner where the walls of the room 10 abut. For example, the angle information for the room 10 may be information represented by an azimuth angle. For example, the angle information may indicate an angle between 0 degrees and 360 degrees according to the clockwise direction with respect to the north (0 degrees).

The device 100 may generate data indicating the structure of the room 10 using the estimated distance information and the obtained angle information for the room 10. For example, the device 100 may provide it to a user by generating and displaying a floor plan showing the structure of the room 10. In addition, the device 100 may generate and provide a 3D model representing the structure of the room 10, and may print the 3D model using a 3D printer. Also, the device 100 may generate a preview model using information about objects located in the room 10. The device 100 may generate and provide a preview model representing the structure of the room 10 by adding information about an object to the generated floor plan or 3D model.

The preview model may be a 3D model including an indoor environment used by a real user. For example, the preview model may include objects such as doors, windows, furniture, wallpaper, and ornaments that may be located indoors.

The device 100 may immediately identify the structure of the room 10 in which the user is located and provide it to the user. Therefore, the user can grasp the structure of the indoor 10 at a glance, and can use the navigation service indoors. Further, the device 100 may provide data indicating the structure of the room 10 to a user who cannot directly grasp the structure of the room 10 such as a visually impaired person in a manner other than a visual method such as an auditory method. have.

2 shows an example in which data indicating the structure of the room is used in the device 100 according to an embodiment.

2(a) shows a map including the area where the building is located. For example, FIG. 2(a) may be a map acquired using GPS. FIG. 2B is a map of the device 100 using data indicating the structure of the room. 2(b) may be a map showing an area as shown in FIG. 2(a).

For example, the device 100 may generate a map of FIG. 2B using data representing the generated indoor structure based on the map of FIG. 2A obtained using GPS. have.

The device 100 may provide data indicating the structure of the room to the user, so that the user can grasp the structure of the building currently located. In addition, the device 100 may provide the user with a map including data indicating the structure of the room, so that the user can easily find the way in the room.

The device 100 may store data indicating the generated indoor structure, and may provide a map including data indicating the indoor structure even when the user moves to another place.

3 shows a hardware configuration of a device 100 that generates data representing a structure of a room according to an embodiment.

The device 100 according to an embodiment includes an audio output unit 310, an audio reception unit 320, a sensing unit 330, and a control unit 350. In addition, the device 100 according to another embodiment may further include a user input unit 340.

In FIG. 3, only hardware components illustrated in FIG. 3 will be described in order to prevent blurring of features of the present embodiment. However, it can be understood by those skilled in the art that other general hardware components may be included in addition to the hardware components illustrated in FIG. 3.

The sound output unit 310 outputs sound data stored in the device 100. The sound output unit 310 may include a speaker, a buzzer, and the like. For example, the sound output unit 310 may include all conventional loudspeakers usable in the portable device 100, but is not limited thereto and may include a parametric speaker.

For example, the sound output unit 310 may output a predetermined sound signal. The sound output unit 310 may output predetermined sound signals sequentially from the first and second positions in the room under the control of the control unit 350. The user may move the device 100 to the first position and the second position, and the sound output unit 310 may sequentially output predetermined sound signals at the first and second positions in the room according to the user input.

The predetermined acoustic signal may be an acoustic energy waveform in a frequency band including a sine sweep, a maximum length sequence (MLS), a frequency in the audible range, or a frequency in the ultrasonic band.

The sound receiving unit 320 is configured to receive sound signals external to the device 100, and may transmit the received sound signals to the control unit 350. The sound receiving unit 320 may include a microphone or the like. For example, the acoustic receiver 320 may include all conventional microphones usable in the portable device 100, but is not limited thereto, and may also include a directional microphone.

For example, the sound receiving unit 320 may receive a reflection signal for a predetermined sound signal. The acoustic receiver 320 may receive the first reflected signal for the predetermined acoustic signal output at the first position at the first position. Also, the acoustic receiver 320 may receive a second reflected signal for a predetermined acoustic signal output at the second position at the second position.

The first reflected signal and the second reflected signal may be reflected signals that are output from the sound output unit 310 and reflected on at least one wall or object in the room to return to the sound receiver 320.

The sound receiving unit 320 may obtain a first reflected signal or a second reflected signal by receiving a continuous sound signal received for a predetermined time from when the predetermined sound signal is output from the sound output unit 310.

The sensing unit 330 is a configuration for acquiring angular information about the room, and may be a configuration including various sensors for sensing the state of the device 100.

The sensing unit 330 may include a camera sensor capable of photographing an external image of the device 100. The sensing unit 330 may transmit the captured image to the control unit 350 or a display unit (not shown).

In addition, the sensing unit 330 is a configuration day including a terrestrial magnetism sensor, a gyroscope sensor, an acceleration sensor, a gravity sensor, a tilt sensor, and the like. Can be.

For example, the sensing unit 330 may acquire at least one angle information about the room using a camera sensor and a geomagnetic sensor.

The user input unit 340 may be a hardware configuration that inputs information for controlling the device 100 from a user. For example, the user input unit 340 may be implemented as a key pad, a dome switch, a touch pad, a jog wheel, a jog switch, and the like.

The user input unit 340 may receive a user input for estimating distance information about the room from the user. For example, the user input unit 340 may receive a user input for outputting a predetermined sound signal at the first position. Also, the user input unit 340 may receive a user input for outputting a predetermined sound signal at the second position.

In addition, the user input unit 340 may receive a user input for obtaining angle information about the room.

In addition, the user input unit 340 may receive user input for generating data representing various indoor structures.

The control unit 350 is a configuration for controlling the overall operation of the device 100, and may include, for example, a central processing unit (CPU).

The control unit 350 controls the sound output unit 310 to output a predetermined sound signal at a first position in the room, and the sound receiving unit 320 to receive a first reflected signal for the output sound signal. Can be controlled. For example, the control unit 350 controls the sound output unit 310 to output a predetermined sound signal when a user input is received at a first position in the room, and receives a first reflected signal when a predetermined sound signal is output. The acoustic receiver 320 can be controlled to do so.

In addition, the control unit 350 may control the sound output unit to output a predetermined sound signal at a second position in the room, and control the sound reception unit to receive a second reflected signal for the output sound signal. For example, the control unit 350 controls the sound output unit 310 to output a predetermined sound signal when a user input is received at a second position in the room, and receives a second reflected signal when a predetermined sound signal is output. The acoustic receiver 320 can be controlled to do so.

In addition, the control unit 350 may estimate distance information about the room using the received first and second reflected signals. For example, the first reflected signal and the second reflected signal may be reflected signals that are reflected by at least one wall of the room 10 and returned to the device 100 by a predetermined acoustic signal output from the device 100, The distance information for the room 10 may be information indicating a distance from at least one wall of the room 10 reflecting a predetermined acoustic signal to the device 100. Further, the distance information may be information indicating a distance from at least one object located in the room 10 reflecting a predetermined acoustic signal to the device 100.

Specifically, the controller 350 may post-process the received first and second reflected signals to estimate distance information about the room.

For example, the controller 350 may de-convolve the first reflected signal and the second reflected signal to generate respective impulse responses. The controller 350 may de-convolve the first reflected signal from the predetermined acoustic signal to generate an impulse response of the first reflected signal, and deconvolute the second reflected signal from the predetermined acoustic signal By doing so, the impulse response of the second reflected signal can be generated.

In addition, the control unit 350 may filter the impulse response of the first reflected signal and the impulse response of the second reflected signal to reduce noise. Also, the controller 350 may auto-correlate the impulse response of the first reflected signal and the impulse response of the second reflected signal, respectively, to reduce noise and emphasize the reflected signal.

The controller 350 may extract a plurality of peaks from the post-processed first reflected signal and second reflected signal, respectively. For example, a plurality of peaks may appear to be arranged close to each other on the time axis, and each peak may be clustered to represent one acoustic reflection signal.

The control unit 350 may map the first reflected signal and the second reflected signal extracted as peaks based on the distance between the first position and the second position, and estimate distance information for the room based on the mapping Can be.

Specifically, the controller 350 maps the second reflection signal extracted as peaks with the time axis by moving a time axis based on the time required for the predetermined sound signal to move the distance between the first position and the second position. can do. The distance between the first position and the second position is a distance that the user moves the device 100 and may be measured by the control unit 350 when moving. The speed of the predetermined sound signal is generally the speed of the sound signal in air at room temperature, and may be predetermined at about 340 m/s. Accordingly, the controller 350 may calculate the time required to move the distance between the first position and the second position using a predetermined speed of the predetermined acoustic signal. The speed of the acoustic signal may be determined differently depending on the indoor environment.

The controller 350 may estimate distance information for the room by mapping the first reflection signal and the second reflection signal shifted in time axis. Specifically, the controller 350 may map peaks mapped at the same time among peaks of the first reflection signal and peaks of the second reflection signal shifted in time axis.

The controller 350 may estimate information indicating a distance to at least one wall or object in the room by using a signal of a peak closest in time among peaks mapped at the same time. The controller 350 may estimate distance information for the room using the peak time of the signal closest in time and the speed of a predetermined acoustic signal. The controller 350 may determine signals of the remaining peaks except for the closest temporally among the peaks mapped at the same time as a noise signal.

In addition, the control unit 350 may control the sensing unit 330 to acquire at least one of an angle with respect to at least one wall in the room or an angle with respect to a corner where the walls in the room contact. For example, the angle to the wall or the angle to the corner may be an azimuth angle using a geomagnetic sensor.

For example, when the device 100 acquires a user input for obtaining angle information from a user while the device 100 faces at least one wall in the room, the sensing unit 330 obtains an angle with respect to the wall Can be controlled. In addition, the controller 350 may control the sensing unit 330 to acquire an angle with respect to the corner when the device 100 acquires user input while facing the corner where the walls of the room are in contact. The user may use the camera lens and display unit (not shown) of the sensing unit 330 to direct the device 100 to the wall or corner.

The control unit 350 may generate data indicating the structure of the room using the estimated distance information of the room and the obtained at least one angle information. For example, the control unit 350 may generate a floor plan showing the structure of the room, or may generate a 3D model showing the structure of the room. In addition, the control unit 350 may generate a preview model indicating the structure of the room by adding data indicating the structure of the object to the 3D model using information about the objects located in the room.

4 to 7 illustrate a process of estimating distance information for the room 40 using a predetermined sound signal in the device 100 according to an embodiment.

Specifically, the device 100 performs a process for estimating the distance d 411 from the device 100 located at the first location 410 to the wall 41 of the room 40.

4 illustrates a process of obtaining reflection signals for a predetermined sound signal in the device 100 according to an embodiment.

In FIG. 4A, the device 100 may output a predetermined sound signal at the first position 410 of the room 40. The output predetermined acoustic signal may be reflected by the wall 41 of the room 40, and the device 100 may receive the first reflected signal reflected by the wall 41 at the first position 410. Can be.

In addition, the output of the predetermined acoustic signal may be reflected by other walls or objects of the room 40 other than the wall 41 to generate a plurality of reflected signals. Accordingly, the device 100 receives a continuous sound signal received for a predetermined time from when the predetermined sound signal is output at the first position 410 to receive the first reflected signal including the plurality of reflected signals. Can be. For example, the first reflected signal may appear as a graph 412. The X-axis of the graph 412 may be time and the Y-axis may be the magnitude of the signal.

In FIG. 4B, the device 100 may output a predetermined sound signal at the second position 420 of the room 40. The device 100 may be moved from the first location 410 of the room 40 to the second location 410 by the user. The predetermined acoustic signal output from the second position 420 may be reflected by the wall 41 of the room 40, and the device 100 may be reflected by the wall 41 at the second position 420. The second reflected signal may be received.

In addition, the output of the predetermined acoustic signal may be reflected by other walls or objects of the room 40 other than the wall 41 to generate a plurality of reflected signals. Accordingly, the device 100 receives a continuous acoustic signal received for a predetermined time from when the predetermined acoustic signal is output at the second position 420 to receive the second reflected signal including the plurality of reflected signals. Can be. For example, the second reflected signal may appear as a graph 422. The X-axis of the graph 422 may be time and the Y-axis may be the magnitude of a signal.

5 illustrates a process of post-processing reflection signals obtained from the device 100 according to an embodiment.

Specifically, the first reflection signal may be post-processed by the device 100. For example, the device 100 may deconvolve the first reflected signal from a predetermined acoustic signal to generate an impulse response of the first reflected signal, and filter and autocorrelate the impulse response of the first reflected signal. have. The device 100 may extract a plurality of peaks from the post-processed first reflected signal and display the first reflected signal as a graph 414. Each of the peaks shown in the graph 414 may represent acoustic reflection signals for a predetermined acoustic signal output at the first position 410. The X-axis of the graph 414 is time and the Y-axis may be the magnitude of the impulse response of the first reflected signal.

Also, the second reflected signal may be post-processed by the device 100 in the same manner as the first reflected signal. The device 100 may extract a plurality of peaks from the post-processed second reflected signal and display the second reflected signal as a graph 424. Each of the peaks shown in the graph 424 may represent acoustic reflection signals for a predetermined acoustic signal output at the second position 420. The X-axis of the graph 424 is time and the Y-axis may be the magnitude of the impulse response of the second reflected signal.

6 and 7 illustrate a process of mapping reflected signals obtained from the device 100 according to an embodiment.

Referring to FIG. 6, the device 100 may measure a distance d 430 between the first location 410 and the second location 420. For example, when the user moves from the first location 410 to the second location 420, the device 100 uses an acceleration sensor or GPS to move between the first location 410 and the second location 420. The distance d 430 can be measured.

The device 100 may move the second reflected signal on the time axis using the measured distance d 430. Specifically, the device 100 may move the second reflected signal on the time axis according to the time required for the predetermined sound signal to move the distance d 430. The device 100 may calculate a time required for the predetermined sound signal to move the distance d 430 using the predetermined speed of the predetermined sound signal. The second reflected signal moved in the time axis may appear as a graph 426.

Alternatively, the device 100 may move the first reflected signal on the time axis instead of moving the second reflected signal on the time axis. When the first reflection signal is moved in the time axis, the device 100 may estimate the distance d 421 from the device 100 located at the second position 420 to the wall 41 of the room 40. have.

Referring to FIG. 7, the device 100 may map the first reflected signal shown in the graph 414 and the second reflected signal moved in the time axis shown in the graph 426. Specifically, the device 100 may map peaks mapped at the same time among peaks of the first reflection signal and peaks of the second reflection signal shifted in time axis. In FIG. 6, the peaks of the first reflected signal at t1 and t2 and the peaks of the second reflected signal shifted in time axis may be mapped.

The device 100 may estimate a distance 411 from the device 100 to the wall 41 by using a signal of a peak closest in time among peaks mapped at the same time. Specifically, the device 100 may estimate the distance 411 using the time t1 of the closest peak in time and the speed of a predetermined acoustic signal.

8 is a diagram for explaining a process of obtaining angle information about a room in the device 100 according to an embodiment.

The user may obtain at least one angle information for the room 80 using the device 100 in the room 80. The angle information for the room 80 may be information indicating an angle with respect to at least one wall of the room 80 or an angle with respect to a corner of the walls of the room 80. For example, the angle information for the room 80 may be information represented by an azimuth angle.

Referring to FIG. 8, the user can direct the device 100 toward at least one wall of the room 80, and the device 100 may obtain angle information about the wall. In addition, the user can direct the device 100 toward at least one corner of the room 80, and the device 100 may acquire angle information for the corner.

In order to obtain more accurate angle information, the device 100 may photograph the indoor 80 in a direction facing the device 100 using a camera sensor, and provide an image captured to a user using a display unit can do. The user can obtain more accurate angle information by checking the position of the wall or corner by using the image of the displayed indoor 80 and the auxiliary line 810 provided on the display. The auxiliary line 810 is a solid line or a grid line displayed on a screen on which an image is displayed, and may be a line fixedly displayed regardless of the movement of the image.

The device 100 may continuously acquire a plurality of angle information, and may store the obtained plurality of angle information.

9 is a view for explaining a process of generating data representing the structure of the interior 90 in the device 100 according to an embodiment.

The device 100 may generate data indicating the structure of the room 90 using distance information and angle information for the room 90. For example, the device 100 may generate a plan view showing the structure of the interior 90 as shown in FIG. 9.

The device 100 may estimate a distance d 910 from the location 901 of the device 100 to at least one wall 91 of the room 90 using reflection signals for a predetermined acoustic signal. . Also, the device 100 may acquire at least one angle information about the room 90. For example, the device 100 may include angular information on at least one of the walls 91, 92, 93, and 94 of the interior 90, or a first corner, a second corner, a third corner of the interior 90, and At least one of the angle information for the fourth corner may be obtained. The device 100 may obtain angle information as information indicating an azimuth angle based on the position 901 of the device 100.

The device 100 may generate a plan view showing the structure of the room 90 based on the obtained angle information and the estimated distance information d 910. The device 100 may display a floor plan showing the structure of the generated room 90 and provide it to the user.

The device 100 may store a floor plan, which is data representing the generated structure of the room 90, and continuously provide a map including data indicating the structure of the room 90 even when the user moves to another place. have.

10 shows an example of a user interface screen provided to generate data representing the structure of the room 90 in the device 100 according to an embodiment.

Referring to 1010, the device 100 may measure a user's movement distance. For example, when the user moves from the first location to the second location with the device 100, the device 100 may measure the user's movement distance using an acceleration sensor or GPS. Alternatively, the device 100 may estimate the moving distance of the user by measuring the number of steps of the user.

Also, the device 100 may track a user's movement path. For example, when the user moves from the first location to the second location with the device 100, the device 100 may track and display the user's movement path using a geomagnetic sensor or GPS. . The device 100 may determine relative positions between different rooms by tracking a user's movement path.

The device 100 may provide various user interfaces, such as 1010, to estimate a user's moving distance or moving path.

Referring to 1020, the device 100 may display data indicating the generated indoor structure. For example, the device 100 may display a floor plan showing the structure of the room. The device 100 may display values indicating dimensions of a plan view together.

Also, the device 100 may provide a user interface for modifying data representing the generated indoor structure. The user may modify the displayed floor plan using the provided user interface. The user may input a name for the room using the user interface.

Referring to 1030, the device 100 may display data representing the generated structures of a plurality of rooms. The device 100 may generate data representing the structure of a plurality of rooms and merge them to generate data representing the structure of one entire room. For example, the device 100 may generate data representing the structure of the entire house by generating and merging respective data for rooms in the house.

The device 100 may merge data by tracking a user's movement path. Also, the device 100 may merge data using angle information for the room.

In addition, the device 100 may generate data indicating the structure of the entire room for a plurality of floors by merging data indicating the structure of the room for each floor.

The device 100 may display data indicating the structure of the entire merged room and provide it to the user. In addition, the device 100 may provide a user interface for modifying data representing the entire indoor structure. The user may modify the displayed data using the provided user interface.

The device 100 may store data indicating the structure of the generated room, and provide the generated data to a navigation service to receive a navigation service for the room.

11 is a diagram for explaining a method of generating data representing a structure of a room in the device 100 according to an embodiment.

The device 100 may generate data indicating the structure of the room and provide it to the user. For example, the data representing the structure of the room may be a plan view 1110 of the room, a three-dimensional model 1120 or a preview model 1130. Alternatively, the device 100 may generate data representing the structure of the room as auditory data, not visual data, and provide it to the user.

The device 100 may generate a plan view 1110 showing the structure of the room using the estimated distance and angle information for the room. The device 100 may generate data indicating the structure of the entire room by merging the generated data.

In addition, the device 100 may generate a 3D model 1120 showing the structure of the room using the generated floor plan 1110. For example, the user may point the device 100 toward the ceiling, and the device 100 may measure distance information from the device 100 to the ceiling. Alternatively, the device 100 may acquire angle information about a corner between at least one wall and a ceiling in the room.

The device 100 may generate a 3D model 1120 representing the structure of the room based on the floor plan 1110, using distance information to the ceiling or angle information about a corner between the wall and the ceiling.

The device 100 may display the generated 3D model 1120 and provide it to the user, or may provide the 3D printer to print the 3D model.

In addition, the device 100 may generate a preview model 1130 showing the structure of the room using the generated floor plan 1110 or the 3D model 1120. The preview model 1130 may be a 3D model including an indoor environment used by an actual user. For example, the preview model 1130 may include objects such as doors, windows, furniture, wallpaper, and ornaments that may be located indoors.

The device 100 may provide information about various objects that may be included in the preview model 1130 to the user. For example, the device 100 may provide the user with information on the name, dimensions, or image information of objects, and may receive a user input for selecting objects to be included in the preview model 1130 from the user.

Also, the device 100 may directly obtain and use information about an object located in the room. The device 100 may obtain dimension information about an object located in the room and use it to generate the preview model 1130. Specifically, the dimensional information for the object may be estimated in the same way as the method for estimating the distance information for the room described above with reference to FIGS. 4 to 7.

The device 100 may generate the preview model 1130 by merging the generated plan view 1110 or the 3D model 1120 with information about selected objects. For example, the device 100 may superimpose the images of the selected objects 1132 and 1134 on the 3D model 1120 and generate and display the preview model 1130.

The device 100 may scale the selected object according to the dimension information of the selected object and add it to the preview model 1130. Accordingly, the user can check in advance the actual structure of the room including the selected object through the preview model 1130.

12 is a flowchart of a method for generating data representing a structure of a room in the device 100 according to an embodiment.

Referring to FIG. 12, in operation 1210, the device 100 may output a predetermined acoustic signal at a first position and a second position indoors, and acquire a first reflected signal and a second reflected signal, respectively.

The first reflection signal and the second reflection signal may be reflection signals from which a predetermined sound signal output from the device 100 is reflected on at least one wall in the room and returned to the device 100. The user sequentially moves the device 100 to the first position and the second position, so that the device 100 sequentially outputs a predetermined acoustic signal at the first position to obtain the first reflected signal, and the second position to the predetermined position. The second acoustic signal may be obtained by outputting the acoustic signal of.

In operation 1220, the device 100 may estimate distance information about the room using the obtained first and second reflected signals.

The indoor distance information may be information indicating a distance from at least one wall in the room reflecting a predetermined acoustic signal to the device 100.

The device 100 may post-process the first reflected signal and the second reflected signal to estimate distance information about the room.

For example, the device 100 may de-convolve the first reflected signal and the second reflected signal to generate respective impulse responses. The device 100 may de-convolve the first reflected signal from the predetermined acoustic signal to generate an impulse response of the first reflected signal, and deconvolve the second reflected signal from the predetermined acoustic signal By doing so, the impulse response of the second reflected signal can be generated.

Further, the device 100 may filter the impulse response of the first reflected signal and the impulse response of the second reflected signal to reduce noise. Further, the device 100 may auto-correlate the impulse response of the first reflected signal and the impulse response of the second reflected signal, respectively, to reduce noise and emphasize the reflected signal.

Also, the device 100 may extract a plurality of peaks from the post-processed first reflected signal and second reflected signal, respectively. For example, a plurality of peaks may appear to be arranged close to each other on the time axis, and each peak may be clustered to represent one acoustic reflection signal.

The device 100 may map the first reflected signal and the second reflected signal extracted as peaks based on the distance between the first position and the second position, and estimate distance information for the room based on the mapping You can.

Specifically, the device 100 moves the second reflected signal extracted as peaks in time axis according to the time required for the predetermined sound signal to move the distance between the first position and the second position, and maps the first reflected signal. can do. The distance between the first position and the second position is a distance that the user moves the device 100 and can be measured by the device 100 when moving.

The device 100 may estimate distance information about the room by mapping the first reflection signal and the second reflection signal shifted in time axis. Specifically, the device 100 may map peaks mapped at the same time among peaks of the first reflection signal and peaks of the second reflection signal shifted in time axis.

The device 100 may estimate information indicating a distance to at least one wall or object in the room by using a signal of a peak closest in time among peaks mapped at the same time. The device 100 may estimate distance information about the room using the peak time of the signal closest in time and the speed of a predetermined acoustic signal.

In operation 1230, the device 100 may acquire at least one angle information about the room.

The angle information for the room may be information indicating an angle with respect to at least one wall in the room or an angle with respect to a corner where the walls in the room contact. For example, the angle information for the room may be information represented by an azimuth angle.

The user may direct the device 100 toward at least one wall in the room, and the device 100 may obtain angle information about the wall. In addition, the user can direct the device 100 toward at least one corner of the room, and the device 100 may acquire angle information for the corner.

In order to obtain more accurate angle information, the device 100 may photograph the indoor 80 in a direction facing the device 100 using a camera sensor, and provide an image captured to a user using a display unit can do.

The device 100 may continuously acquire a plurality of angle information, and may store the obtained plurality of angle information.

In operation 1240, the device 100 may generate a floor plan of the room showing the structure of the room using the estimated distance information and the obtained angle information.

For example, the device 100 may provide the user by generating and displaying a floor plan showing the structure of the room. Further, the device 100 may generate and provide a 3D model representing the structure of the room, and may print the 3D model using a 3D printer.

In addition, the device 100 may generate a preview model indicating the structure of the room by using information about objects located in the room. The device 100 may generate and provide a preview model representing the structure of the room by adding information about the object to the generated floor plan or 3D model.

13 illustrates a hardware configuration of a device 1300 for generating data representing the structure of a room according to an embodiment.

Referring to FIG. 13, the device 1300 may include an input unit 1310, an output unit 1320, a communication unit 1330, a sensing unit 1340, a memory 1350, and a control unit 1360.

In the device 1300, the input unit 1310 may include a touch recognition module 1311, a motion recognition module 1312, a key recognition module 1312, and a sound receiving module 1313, and the output unit 1320 is It may include a display unit 1321, an audio output module 1322 and an alarm unit 1323, the communication unit 1330 is a short-range communication module 1331, a wireless Internet module 1332, a mobile communication module 1333 and Wired Internet module 1334 may be included. In FIG. 13, only hardware components related to the present embodiment will be described in order to prevent blurring of features of the present embodiment. However, according to the type of the device 1300, other general-purpose hardware components may be further included in addition to the hardware components shown in FIG. 13, or some hardware components shown in FIG. 13 may be omitted. Those skilled in the art can understand.

The method of generating data indicating the structure of the room according to an embodiment may be performed by the device 1300 illustrated in FIG. 13, and the device 100 illustrated in FIG. 3 previously includes the device illustrated in FIG. 13 ( 1300).

The device 100 of FIG. 3 may perform some or all of the functions performed by the device 1300 of FIG. 13. The sound output unit 310 of FIG. 3 may correspond to the sound output module 1322 of FIG. 13, and the sound reception unit 320 of FIG. 3 may correspond to the sound reception module 1313 of FIG. 13, The sensing unit 330 of 3 may correspond to the sensing unit 1340 of FIG. 13, and the user input unit 340 of FIG. 3 may correspond to the touch recognition module 1311 and the key recognition module 1312 of FIG. 13. The control unit 350 of FIG. 3 may correspond to the control unit 1360 of FIG. 13.

Hereinafter, components of the device 1300 of FIG. 13 will be described in order.

The input unit 1310 may refer to a hardware configuration in which a user inputs information for controlling the device 1300. For example, the input unit 1310 may be implemented as a key pad, a dome switch, a touch pad, a jog wheel, a jog switch, and the like.

The touch recognition module 1311 may detect a user's touch gesture or touch input, and transmit information regarding the sensed touch gesture or touch input to the controller 1360.

The touch recognition module 1311 may include various sensors for detecting a touch or a proximity touch. To detect touch gestures or touch inputs in the touch recognition module 1311, sensors such as a contact-capacitive method, a pressure-resistant resistive method, an infrared detection method, a surface ultrasonic conduction method, an integral tension measurement method, or a piezo effect method Can be implemented as

Also, the touch recognition module 1311 may detect a proximity touch using a proximity sensor. The proximity sensor refers to a sensor that detects the presence or absence of an object approaching a predetermined detection surface or an object in the vicinity without mechanical contact using electromagnetic force or infrared rays. Examples of the proximity sensor include a transmission type photoelectric sensor, a direct reflection type photoelectric sensor, a mirror reflection type photoelectric sensor, a high frequency oscillation type proximity sensor, a capacitive type proximity sensor, a magnetic type proximity sensor, and an infrared proximity sensor.

The user's touch gesture or touch input may include tap, touch & hold, double tap, drag, flick, drag & drop, hovering, and the like.

The touch recognition module 1311 may be implemented as a touch screen by forming a plurality of layer structures with the display unit 1321.

The key recognition module 1312 may recognize a user's command input through a hardware key (eg, direction key, letter key, mouse, etc.).

The sound receiving module 1313 is a configuration for receiving an external sound signal, and may transmit the received sound signal to the controller 1360.

The output unit 1320 is a hardware configuration for outputting an audio signal, a video signal, or an alarm signal.

The display unit 1321 is display interfacing means for displaying various types of information, such as information processed by the device 1300 or information to be processed, to the user. The display unit 1321 may display a graphical user interface (GUI) for visually and intuitively providing information processed by the device 1300 to a user. The display unit 1321 may perform the function of the display unit 130 illustrated in FIG. 12. Meanwhile, the display unit 130 of FIG. 12 may be implemented as a hardware structure of a touch screen including the touch recognition module 1311 and the display unit 1321.

The sound output module 1322 outputs audio data received through the communication unit 1330 or stored in the memory 1350. The sound output module 1322 may include a speaker, a buzzer, and the like.

The alarm unit 1323 outputs a signal for notifying the event generated in the device 1300. The alarm unit 1123 may output a signal for notifying the occurrence of an event in at least one of an audio signal, a video signal, and a vibration signal.

The communication unit 1330 is a hardware configuration having a function of communicating with an external network or an external device.

The short-range communication module 1331 refers to a module for short-range communication. Bluetooth (Bluetooth), UWB (Ultra Wideband), ZigBee (ZigBee), NFC (Near Field Communication), WFD (Wi-Fi Direct), infrared communication (IrDA, infrared data association) may be used as a short-range communication technology. .

The wireless Internet module 1332 is a module for wireless Internet access. The mobile communication module 1333 is a module for communicating with a mobile communication network. The wired Internet module 1334 is a module for wired Internet access.

The sensing unit 1340 may be a hardware configuration for sensing the movement of the device 1300 or the environment around the device 1300. For example, the sensing unit 1340 includes a terrestrial magnetism sensor, a gyroscope sensor, an acceleration sensor, a gravity sensor, a tilt sensor, etc. can do. In addition, the sensing unit 1340 may include a camera, an infrared camera, a proximity sensor, and the like.

The memory 1350 is a hardware configuration that stores information processed by the device 1300, and includes a hard-disk drive (HDD), a solid state drive (SDD), random access memory (RAM), and read only memory (ROM). Can be implemented. For example, the memory 1350 may store overall information regarding a user interface to be displayed through the display unit 1321.

The control unit 1360 is a hardware configuration that controls the overall operation and functions of the device 1300, and may be implemented by at least one processor such as a CPU and an AP.

The device according to the present invention includes a processor, a memory for storing and executing program data, a permanent storage such as a disk drive, a communication port communicating with an external device, a user interface such as a touch panel, keys, buttons, etc. Devices and the like. Methods implemented by a software module or algorithm may be stored on a computer-readable recording medium as computer-readable codes or program instructions executable on the processor. Here, as a computer-readable recording medium, a magnetic storage medium (eg, read-only memory (ROM), random-access memory (RAM), floppy disk, hard disk, etc.) and optical reading medium (eg, CD-ROM (CD-ROM) ), DVD (Digital Versatile Disc). The computer-readable recording medium can be distributed over network coupled computer systems so that the computer readable code is stored and executed in a distributed fashion. The medium is readable by a computer, stored in memory, and can be executed by a processor.

All documents, including published documents, patent applications, patents, etc., cited in the present invention may be incorporated into the present invention in the same way that each cited document is individually and specifically merged or represented as a whole in the present invention. .

For understanding of the present invention, reference numerals have been described in preferred embodiments illustrated in the drawings, and specific terms are used to describe embodiments of the present invention, but the present invention is not limited by the specific terms, and the present invention Can include all of the components that are ordinarily considered by those skilled in the art.

The present invention can be represented by functional block configurations and various processing steps. These functional blocks can be implemented with various numbers of hardware or/and software configurations that perform specific functions. For example, the present invention is directed to integrated circuit configurations such as memory, processing, logic, look-up tables, etc., capable of executing various functions by control of one or more microprocessors or other control devices. Can be employed. Similar to the components of the present invention that can be executed with software programming or software elements, the present invention includes C, C++, including various algorithms implemented with a combination of data structures, processes, routines or other programming configurations. , Can be implemented in programming or scripting languages such as Java, assembler, etc. Functional aspects can be implemented with algorithms running on one or more processors. In addition, the present invention may employ conventional techniques for electronic environment setting, signal processing, and/or data processing. Terms such as “mechanism”, “element”, “means”, and “configuration” can be used broadly and are not limited to mechanical and physical configurations. The term may include the meaning of a series of routines of software in connection with a processor or the like.

The specific implementations described in the present invention are exemplary embodiments, and do not limit the scope of the present invention in any way. For brevity of the specification, descriptions of conventional electronic configurations, control systems, software, and other functional aspects of the systems may be omitted. In addition, the connection or connection members of the lines between the components shown in the drawings are illustrative examples of functional connections and/or physical or circuit connections. In the actual device, alternative or additional various functional connections, physical It can be represented as a connection, or circuit connections. In addition, unless specifically mentioned, such as “essential”, “importantly”, etc., it may not be a necessary component for application of the present invention.

In the specification (especially in the claims) of the present invention, the use of the term “above” and similar indication terms may be in both singular and plural. In addition, in the case of describing a range in the present invention, as including the invention to which the individual values belonging to the range are applied (if there is no contrary description), each individual value constituting the range is described in the detailed description of the invention. Same as Finally, unless there is a clear or contradictory description of the steps constituting the method according to the invention, the steps can be done in a suitable order. The present invention is not necessarily limited to the description order of the above steps. The use of all the examples or exemplary terms (eg, etc.) in the present invention is merely for describing the present invention in detail, and the scope of the present invention is limited due to the above examples or exemplary terms unless it is defined by the claims. It does not work. In addition, those skilled in the art can recognize that various modifications, combinations, and changes can be configured according to design conditions and factors within the scope of the appended claims or equivalents thereof.

Claims (21)

A method for generating data representing a structure of a room in a device,
Outputting a predetermined sound signal at a first position and a second position in the room, and obtaining a first reflection signal and a second reflection signal for the output predetermined sound signal, respectively;
Estimating distance information for the room by mapping the obtained first and second reflected signals based on the distance between the first position and the second position;
Obtaining, by the device, at least one angle information for the room; And
And generating a plan view of the room representing the structure of the room using the estimated distance information and the obtained angle information. According to claim 1,
And generating a 3D model representing the structure of the room using the generated floor plan. According to claim 1,
And generating a three-dimensional model representing the structure of the room by using the generated floor plan and information on objects located in the room. According to claim 1,
The distance information for the room is information indicating a distance from the device to at least one of the walls of the room or objects located in the room. According to claim 1,
Estimating the distance information for the room,
And estimating distance information for the room based on the distance between the first position and the second position. According to claim 1,
Obtaining an image of the interior; And
Including; displaying at least one auxiliary line on the image of the room;
The corner and wall of the room are obtained based on the image of the room and the at least one auxiliary line. According to claim 1,
Mapping the first reflected signal and the second reflected signal,
A method of mapping the first reflection signal and the second reflection signal using a time required for the predetermined sound signal to move a distance between the first position and the second position. According to claim 1,
The first reflected signal and the second reflected signal are acoustic signals reflected by the predetermined acoustic signal to at least one wall or object in the room and returned to the device,
Estimating the distance information for the room,
And estimating information representing the distance from the device to at least one wall or object in the room. According to claim 1,
The obtaining of at least one angle information for the room may include:
And obtaining at least one of an angle with respect to at least one wall of the room or an angle with respect to a corner where the walls of the room abut. According to claim 1,
The obtaining of at least one angle information for the room may include:
Obtaining at least one angle information for the room based on a user input to at least one wall or at least one corner of the room; And
And storing the obtained at least one angle information. Sound output unit;
An acoustic receiver;
A sensing unit that the device acquires at least one angle information for the room; And
At the first position in the room, the sound output unit is controlled to output a predetermined sound signal, and the sound receiver is controlled to receive a first reflected signal for the output sound signal
A control unit for controlling the sound output unit to output the predetermined sound signal at a second position in the room, and controlling the sound reception unit to receive a second reflected signal for the output sound signal,
The control unit,
Mapping the received first reflected signal and the second reflected signal based on the distance between the first position and the second position to estimate the distance information for the room,
And generating a plan view of the room representing the structure of the room using the estimated distance information and the obtained angle information. The method of claim 11,
The control unit,
A device that generates a three-dimensional model representing the structure of the room using the generated floor plan. The method of claim 11,
The control unit,
A device for generating a three-dimensional model representing the structure of the room by using the generated floor plan and information about objects located in the room. The method of claim 11,
The distance information for the indoor device is information indicating a distance from the device to at least one of the indoor walls or the indoor objects. The method of claim 11,
The control unit,
A device for estimating distance information for the room based on the distance between the first position and the second position. The method of claim 11,
Further comprising a display unit,
The sensing unit acquires the image of the interior,
The control unit controls a display unit to display at least one auxiliary line on the indoor image and the indoor image,
The corner and wall of the room are devices obtained based on the image of the room and the at least one auxiliary line. The method of claim 11,
The control unit,
A device for mapping the first reflection signal and the second reflection signal using a time required for the predetermined sound signal to move a distance between the first position and the second position. The method of claim 11,
The receiving unit,
Obtaining the first reflection signal and the second reflection signal, which are acoustic signals reflected from at least one wall or object in the room and returned to the device;
The control unit,
A device that estimates information indicating a distance from the device to at least one wall or object in the room.
delete The method of claim 11,
Further comprising a user input unit for obtaining a user's input to the at least one wall or at least one corner of the room,
The sensing unit acquires at least one angle information about the room based on the obtained user input. A computer-readable recording medium recording a program for executing the method of any one of claims 1 to 10 on a computer.

Images