KR102329785B1 - Method for estimating head direction of terminal for position measurement - Google Patents

Abstract

The present invention relates to a method for estimating a heading direction of a terminal for position measurement, a method for a computing device operating by at least one process to estimate a heading direction, and to collect heading direction values from a geomagnetic sensor for a predetermined time, and , collecting images from the image sensor, calculating a plurality of directional changes between the collected images, calculating a plurality of reliability ranges to which the calculated directional changes are applied to heading direction values, calculating a reliability score by counting cases included in the reliability ranges of the dog, and setting a heading direction value having a highest value among the calculated reliability scores as a reference heading direction value.

Description

Method of estimating the heading direction of the terminal for position measurement

The present invention relates to a method for estimating a heading direction of a terminal for location measurement.

In general, indoor location is estimated by estimating the absolute location or estimating the degree of change in location. As a method of estimating the absolute location, a wireless network or an image or radio wave map is used.

For example, in the case of using a wireless network, the location of the signal source is set as a reference point, the distance is estimated based on time or the degree of loss of signal strength, and then the location of the terminal is estimated. Alternatively, in the case of using an image or a radio wave map, positioning is performed indoors based on information collected in advance for each arbitrary location in the room.

However, in this method of estimating the location, a delay time occurs because the location is estimated by performing comparative analysis after receiving external information without directly utilizing the sensor value of the terminal, and it is difficult to continuously accurately estimate the location. In detail, in the case of the electromagnetic wave map-based positioning technology, the database is configured in grid units, and since the location is estimated based on the representative grid, the separation distance between the centers of the grid is generated, and discontinuous estimation results are obtained.

On the other hand, as a method of estimating the degree of change in position, a PDR method using geomagnetism, acceleration, gravity sensors, etc. from a sensor of the terminal and a method utilizing a change in a camera-based image are used.

The PDR method has the advantage of being able to secure very high accuracy for a short period of time, but the accuracy decreases rapidly as time elapses due to an error inherent in the sensor. In particular, in the case of a geomagnetic sensor for locating a terminal, an error value may appear very large due to an indoor environment, so it is difficult to secure reliability. In addition, since it is difficult to directly estimate the change amount of the feature point from the image by using an image sensor to estimate the position, the initial value must be set based on the information on the absolute position change.

Therefore, there is a need for a technique for estimating a heading direction to obtain a reliable relative position estimation result by improving discontinuity without setting a reference position for absolute position estimation in an indoor environment.

An object to be solved is to provide a technology for estimating and correcting a heading direction of a terminal with improved reliability through a combination between positioning information by a geomagnetic sensor and positioning information using an image sensor.

A method for estimating a heading direction by a computing device operating by at least one process according to an embodiment of the present invention, comprising: collecting heading direction values from a geomagnetic sensor for a predetermined time and collecting images from an image sensor; Calculating a plurality of direction changes between the images, calculating a plurality of reliability ranges to which the calculated direction changes are applied to heading direction values, counting cases in which the heading direction value is included in a plurality of reliability ranges calculating a score, and setting a heading direction value having a highest value among the calculated reliability scores as a reference heading direction value.

In the collecting step, images corresponding to the same timing of measuring the heading direction value for a predetermined time may be collected.

Calculating the amount of direction change includes performing a plurality of groupings for calculating the amount of change between images, extracting a center line for each of the grouped images, and determining the movement distance of the extracted center line between the grouped images. The method may further include calculating, and calculating a direction change amount based on the moving distance of the center line.

The step of calculating the movement distance of the extracted center line includes selecting a feature point located on a center line of a reference image from among grouped images, extracting the feature point from the grouped images, and extracting the feature point and the center line of each image. It is possible to calculate the number of pixels in the

In the calculating of the plurality of reliability ranges, the number of grouped reliability ranges may be calculated based on images corresponding to the same timing for each heading direction value.

When the reference heading direction value is set, calculating the direction change amount from images taken at different timings, and applying the calculated direction change amount to the reference heading direction value may further include the step of correcting the heading direction value at the current time. ek.

The setting of the reference heading direction value may further include extending a section for a predetermined time or initializing the reference heading direction value when the calculated confidence score is smaller than a set threshold value.

The method may further include initializing a reference heading direction value when movement of more than a threshold value in the lateral direction or the longitudinal direction is detected.

A method of correcting a heading direction by a computing device operating by at least one process according to an embodiment of the present invention, the method comprising: collecting heading direction values from a geomagnetic sensor for a predetermined time and collecting images from an image sensor; Calculating a plurality of directional changes between the images, calculating reliability scores of heading direction values based on the directional changes, and setting a heading direction value having the highest value among the calculated reliability scores as a reference heading direction value , and when the reference heading direction value is set, calculating a direction change amount from images photographed at different timings, and applying the calculated direction change amount to the reference heading direction value to calculate a heading direction value at the current time point.

According to the embodiment, it is possible to more accurately estimate and correct the heading direction of the terminal by improving reliability while improving discontinuity that may occur in the positioning process.

According to the embodiment, it is possible to estimate the movement path of the terminal without a separate infrastructure at any location, and by providing positioning information according to an accurate heading direction, it is possible to provide a high satisfaction with the application service according to the user's location.

1 is an exemplary diagram illustrating a series of processes for a terminal to measure a location in an indoor space according to an embodiment of the present invention.
2 is a block diagram illustrating a terminal according to an embodiment of the present invention.
3 is a flowchart illustrating a method of estimating a heading direction of a terminal according to an embodiment of the present invention.
4 is an exemplary diagram illustrating a display screen of a terminal for explaining a process of estimating a direction change amount according to an embodiment of the present invention.

Hereinafter, with reference to the accompanying drawings, embodiments of the present invention will be described in detail so that those of ordinary skill in the art to which the present invention pertains can easily implement them. However, the present invention may be embodied in many different forms and is not limited to the embodiments described herein. And in order to clearly explain the present invention in the drawings, parts irrelevant to the description are omitted, and similar reference numerals are attached to similar parts throughout the specification.

Throughout the specification, when a part "includes" a certain element, it means that other elements may be further included, rather than excluding other elements, unless otherwise stated. In addition, terms such as “…unit”, “…group”, and “module” described in the specification mean a unit that processes at least one function or operation, which may be implemented by hardware or software or a combination of hardware and software. have.

1 is an exemplary diagram illustrating a series of processes for a terminal to measure a location in an indoor space according to an embodiment of the present invention.

As shown in FIG. 1 , in order to know the moving direction of the user moving in the indoor space, it may be confirmed based on the heading direction of the terminal 100 carried by the user.

First, as shown in A of FIG. 1 , a reference heading direction value of the terminal 100 is set.

In this case, to know the heading direction of the terminal 100 may be measured using a geomagnetic sensor, the heading direction value measured by the geomagnetic sensor 161 may include a very large error value due to the indoor environment. Therefore, in order to ensure the accuracy of the heading direction value measured by the geomagnetic sensor 161, the terminal 100 calculates a reliability score for the heading direction values measured for a certain time, and sets the high-reliability heading direction as the reference heading direction. can be set.

Accordingly, when the user carrying the terminal 100 moves indoors in a certain direction, the terminal 100 corrects the movement direction value in real time based on the set reference heading direction value (B). In this case, as a method of correcting the moving direction, a more accurate direction change value can be calculated by using an image captured by the image sensor of the terminal 100 , so that the accurate moving direction can be estimated from the reference heading direction value.

When the movement direction value is corrected in this way, the terminal 100 may correct the movement direction value in real time by applying the direction change value calculated from the image taken at the next timing to the corresponding movement direction value.

In other words, the terminal 100 may estimate the moving direction only from the captured image without separately measuring the heading direction value from the geomagnetic sensor.

On the other hand, when it is determined that it is difficult to calculate the direction change value in the image taken due to the user's sudden change of direction, the terminal 100 resets the reference heading direction value in the same way as in A (C).

Since the terminal 100 calculates the direction change value through the image, it is difficult to compare the image photographed at the previous timing and the image photographed at the current time point when the sudden direction change is made. Therefore, the terminal 100 resets the reference heading direction value when it is difficult to calculate the abrupt direction change or direction change value.

And the terminal 100 may correct the real-time movement direction value based on the reset reference heading direction value (D).

Meanwhile, through the same process, the heading direction of the terminal 100 can be checked in order to check the user's location information and direction information to a place to which the user wants to move in an indoor space.

Therefore, it is possible to more accurately estimate the heading direction of the terminal by minimizing the influence of the external environment in the indoor space.

2 is a block diagram illustrating a terminal according to an embodiment of the present invention.

As shown in FIG. 2 , the hardware of the terminal 100 includes a processor 110 , a memory 120 , a storage 130 , a display 140 , a communication interface 150 , and one or more sensors 160 . In addition, hardware such as an input device and an output device may be included. The terminal 100 is a computing device, and refers to a device equipped with various software including an operating system capable of driving a program and equipped with an arithmetic processing capability. For example, it includes various types of terminals such as a personal computer, a handheld computer, a personal digital assistant (PDA), a mobile phone, a smart device, a tablet, and the like.

Also, the terminal 100 may include a robot that automatically processes or operates a set task.

The processor 110 is a device that controls the operation of the computing device, and may be various types of processors 110 that process instructions included in a program. For example, the processor 110 may be a central processing unit (CPU), a micro processor unit (MPU), a micro controller unit (MCU), a graphic processing unit (GPU), or the like.

The memory 120 loads the corresponding program so that the instructions described to execute the operation of the present invention are processed by the processor 110 . The memory 120 may include, for example, a non-volatile solid-state memory device such as a high-speed random access memory (magnetic disk storage device), a flash memory device, or other non-volatile solid-state memory device (non-volatile solid-state memory device). It may include various types of memory such as volatile memory.

The storage 130 stores various data, programs, etc. required for executing the operation of the present invention.

The display 140 is a UI display means and outputs service information or action information provided by the terminal 100 in real time. The display 140 may be an independent device that directly or indirectly outputs or displays the UI, or may be a part of the device.

The communication interface 150 may be a module that provides wired communication, short-distance or long-distance wireless communication, or a combination thereof.

The one or more sensors 160 include a geomagnetic sensor 161 and an image sensor 162 , and may further include an acceleration sensor 163 and a gyro sensor 164 , but the number or type of the sensors 160 is limited. it is not

The geomagnetic sensor 161 provides azimuth information by detecting geomagnetism in response to the heading direction (going direction) of the terminal 100 . The geomagnetic sensor 161 may measure a two-axis or three-axis type, and includes a magneto resistance (MR) sensor, a magneto impedance (MI) sensor, and the like.

The image sensor 162 captures an image of the front in real time in response to the heading direction, and provides the captured image. The image sensor 162 includes all sensors or devices capable of imaging the environment in front, such as a camera and a solid-state imaging device.

The acceleration sensor 163 may measure the speed per unit time in the three-axis (X, Y, Z) directions by using the gravitational acceleration. The acceleration sensor 163 may measure the velocity, change, and momentum change of the object.

The gyro sensor 164 is a sensor capable of measuring angular velocity, and may measure a physical quantity for three axes. The gyro sensor 164 has a high accuracy of measuring the corresponding angular velocity immediately after the motion occurs, but the accuracy is somewhat low due to the generation of a cumulative error after the motion occurs.

The terminal 100 may calculate a heading direction and a moving direction in an indoor space using a heading direction value, an image, an acceleration value, or an angular velocity value measured by the sensor 160 .

Hereinafter, a method for estimating a direction of a terminal for estimating an accurate heading direction even in a limited environment such as an indoor space will be described in detail with reference to FIGS. 3 to 4 .

3 is a flowchart illustrating a method of estimating a heading direction of a terminal according to an embodiment of the present invention.

As shown in FIG. 3 , the terminal 100 collects the heading direction values of the geomagnetic sensor 161 collected for a predetermined time, and collects images taken from the image sensor 162 ( S110 ).

Step S110 is a step of initializing the heading direction value of the terminal 100, and the heading direction values and images are collected for a predetermined time preset in order to set a reference heading direction value.

Here, a certain time is actually a short time corresponding to n seconds, depending on the conditions of the geomagnetic sensor 161 that measures the heading direction value about 10 to 20 times per second or the image sensor 162 having the number of shooting frames per second. Correspondingly, it can be easily changed and designed later by the user.

In this case, the images taken at the timing at which the heading direction value is collected are collected, and the heading direction value and the same number of images may be collected at the same timing.

Next, the terminal 100 calculates each direction change amount between the collected images (S120).

First, since the terminal 100 needs to analyze two images in order to calculate the amount of change, one image may be grouped with other images. For example, when N images are collected, N-1 grouping may be performed through pairing or grouping with N-1 images for one image.

For example, the image of the first timing and the image of the second timing perform pairing, and when it is desired to calculate the amount of change between the image of the first timing and the image of the third timing, the image of the first timing, the image of the second timing, and the image of the third timing are paired. Images can be grouped.

Therefore, N-1 groupings are performed on each of the N images, and the terminal 100 calculates an amount of direction change between the grouped images.

And the terminal 100 extracts a center line from each of the collected images.

In detail, the terminal 100 may vertically extract the center line of the image by selecting the center in the horizontal direction from the image. In this case, the center line is described as a vertical line, but the present invention is not limited thereto, and the center line may be extracted horizontally or diagonally in response to a condition to be applied later.

For example, if the description is based on the first image, the terminal 100 selects a point located on the extracted center line of the first image as the feature point.

In this case, the feature point may mean a point easily recognizable individually in an image through color, shape, brightness, and the like.

Next, the terminal 100 extracts the feature points extracted from the first image from the grouped images. In addition, the terminal 100 may generate a virtual line from the location of each extracted feature point in the grouped images to the center line of the images. That is, the terminal 100 may estimate the distance from the location of the extracted feature point to each center line as the moving distance.

Here, the imaginary line connects in the vertical direction so as to be the closest distance from the feature point to the center line.

On the other hand, if the feature point selected from the first image in the grouped N images is not extracted from the N-th image, the image from which the feature point is finally extracted is searched for.

And a point located on the center line of the image from which the last selected feature point is extracted is selected as the second feature point. Then, the second feature point is extracted from the N-th image.

In this case, the terminal 100 may calculate the amount of change in the direction between the first image and the N-th image in consideration of both the amount of change of the feature point and the second feature point.

4 is an exemplary diagram illustrating a display screen of a terminal for explaining a process of estimating a direction change amount according to an embodiment of the present invention.

4 (a) is a display screen of the terminal 100 showing the first image, (b) is a display screen of the terminal 100 showing the first image and the grouped image.

As shown in (a) of Figure 4, the terminal 100 selects a feature point (B) on _{the center line (A t ) in the first image.} And as shown in (b) of Figure 4, the terminal 100 _{extracts the center line (A t+1} ) in the grouped image, and extracts the feature point (B) for the first image.

Then, when the terminal 100 connects the center line (A _t+1 ) and the feature point (B) in the vertical direction, it is connected to the point C and the feature point (B) of the center line. In this case, the terminal 100 calculates the number of pixels located on the line connecting the point C of the center line and the feature point B in the grouped image.

The terminal 100 estimates _{the amount of change (Δθ k)} in the heading direction based on the number of moved pixels.

For example, when the physical distance on a complementary metal-oxide semiconductor (CMOS) imaged with the image sensor 162 in the terminal 100 is d, Δθ _k is in units of arctan[(pixel size * moving pixel)/d] ] can be calculated using

Here, since the physical distance d and the size of the pixel are values determined by the characteristics of the terminal, they are constant values.

In this way, the terminal 100 may calculate N-1 direction changes between the grouped images for every N images. _{Then, as shown in the arrows (C t} and C _t+1 ) shown in FIGS. 4 (a) and (b), the amount of direction change can be confirmed.

Next, the terminal 100 calculates a plurality of reliability ranges to which the calculated direction change amounts for each heading direction value are applied ( S130 ).

The terminal 100 calculates the reliability range by using the direction change amount calculated in step S120 to calculate the reliability for the heading direction value.

For example, the difference between the first heading direction value measured at the first timing and the second heading direction value measured at the second timing is the amount of direction change calculated from the grouped second image at the first timing and the second timing. It is best to have

If this is expressed as an equation, it is as in the following equation (1).

[Equation 1]

Here, θ _{M, t+1} is the value of the second heading direction, θ _{M, t is the value of the first heading, and Δθt, e I} is the amount of change in the direction between the grouped first and second images. Indicates the error range.

In other words, the terminal 100 may calculate the reliability range in consideration of the error range in the direction change amount calculated from the grouped images.

In this case, the first heading direction value may have a reliability range corresponding to the number of image groupings. Therefore, each heading direction value has a plurality of reliability ranges.

Next, the terminal 100 calculates a reliability score by counting cases included among a plurality of reliability ranges corresponding to the heading direction value ( S140 ).

The terminal 100 may compare the plurality of reliability ranges for each collected heading direction value and separately count only the cases included in the reliability range.

In this case, the terminal 100 may replace the counted number with a confidence score, set a separate confidence score table, and extract and select a confidence score corresponding to the counted number.

Meanwhile, a method of calculating the reliability score of the terminal 100 may be represented by an algorithm as shown in the following table.

[Table 1]

As shown in Table 1, the terminal 100 may calculate a reliability score for each heading direction value by checking a plurality of reliability ranges corresponding to a plurality of heading direction values. The terminal 100 calculates a reliability score using N heading direction values and N images.

Referring to Table 1, the reliability score is repeatedly calculated from 1 to N using the j-th heading direction value and the direction change value between i and j corresponding to the i-th heading direction value for calculating the reliability score.

In addition, when the i-th heading direction value satisfies the reliability range, the final reliability score may be calculated by cumulative counting corresponding to the corresponding score.

In Table 1, k and l are numbers included in N, k means between the i-th image and the j-th image, and l means the number between the i-th image and the j-th image.

Therefore, Δθ _k represents the amount of change in the direction between the i-th image and the j-th image, and the amount of change in direction when i and j are greater or equal to each other may be applied differently.

In other words, Δθ _l means the amount of heading change between two consecutive images. For example, if l=1, it means the amount of change estimated from images 1 and 2, and if l=2, it means the amount of change between images 2 and 3. Therefore, the reliability range can increase from 1 to N-1, and when the range increases, the values generated during the corresponding range must be accumulated and reflected, so the sum of changes within the region is expressed _{as Δθ k using sigma.}

At this time, the possible error range for the amount of change in the heading direction that may occur from the image is not indicated by setting it to 0 for convenience, but it can be calculated including the error range later.

Next, the terminal 100 sets the heading direction value having the highest value among the calculated reliability scores as the reference heading direction value (S150).

The terminal 100 selects the highest value among the reliability scores calculated for each heading direction value. At this time, when it is determined that the heading direction value having the maximum reliability score is lower than the reference value or not accurate, the terminal 100 may increase the previously set predetermined time or return to step S110 to perform initialization.

의 조건을 만족하지 못하는 경우, 선정된 헤딩 방향 값이 정확하지 않을 가능성이 높다고 판단할 수 있다. For example, the terminal 100 has a reliability value of the selected reference heading direction value.

If the condition of is not satisfied, it can be determined that there is a high possibility that the selected heading direction value is not accurate.

Accordingly, the terminal 100 may further collect a plurality of heading direction values and images for reliability accuracy, and may newly calculate a reliability score together with the previously collected heading direction values and images. Alternatively, the terminal 100 may delete the corresponding heading direction value and images and newly collect a plurality of heading direction values and images to calculate a reliability score.

Also, the heading direction estimating apparatus 100 may initialize the heading direction value and return to step S111 to set the reference heading direction value again.

Next, the terminal 100 calculates the amount of direction change by using the center line and the feature point in the images taken at different timings ( S160 ).

After setting the reference heading direction value, the terminal 100 may calculate the direction change amount in real time in response to the image collected through the image sensor 162 . Since the method of calculating the amount of direction change is the same as the method described in step S120 above, the overlapping description will be omitted.

In this case, in addition to the image sensor 162 , the terminal 100 may calculate the amount of direction change through measurement values of sensors such as the acceleration sensor 163 or the angular velocity 164 . Such a configuration can be easily changed and set by a user later.

Then, the terminal 100 calculates a movement direction value by correcting the heading direction value by applying the direction change amount to the reference heading direction value (S170).

Meanwhile, when the terminal 100 detects a sudden movement through the acceleration sensor 163 or the angular velocity 164 , it is determined that it is difficult to calculate the amount of change in the heading direction through the image obtained from the image sensor 162 . Then, the terminal 100 may return to step S110 to reset the heading direction value.

In this way, through the coupling between the positioning information by the geomagnetic sensor 161 and the positioning information using the image sensor 162, the discontinuity is improved without setting a reference position for absolute position estimation in an indoor environment, so that the accurate device with high reliability is obtained. The heading direction can be estimated and corrected.

In addition, it is possible to accurately estimate the movement path of the device without a separate infrastructure at any location performing relatively limited communication, such as an indoor environment.

The embodiment of the present invention described above is not implemented only through the apparatus and method, but may be implemented through a program for realizing a function corresponding to the configuration of the embodiment of the present invention or a recording medium in which the program is recorded.

Although the embodiments of the present invention have been described in detail above, the scope of the present invention is not limited thereto, and various modifications and improved forms of the present invention are also provided by those skilled in the art using the basic concept of the present invention as defined in the following claims. is within the scope of the right.

Claims (12)

A method for a computing device operated by at least one process to estimate a heading direction, the method comprising:
Collecting heading direction values from the geomagnetic sensor for a certain period of time, and collecting images from the image sensor;
calculating a plurality of directional changes between the collected images;
calculating a plurality of reliability ranges for the heading direction values by applying the plurality of direction change amounts to the heading direction values;
calculating a reliability score of each heading direction value by counting the number of reliability ranges including each heading direction value among the plurality of reliability ranges; and
setting a heading direction value having the highest reliability score among heading direction values collected from the geomagnetic sensor as a reference heading direction value
A method of estimating the heading direction, including In claim 1,
The collecting step is
A method of estimating the heading direction by collecting images corresponding to the same timing of measuring heading direction values for a certain period of time. In claim 2,
The step of calculating the direction change amount,
performing a plurality of groupings for calculating the amount of change between the images;
extracting a centerline for each of the grouped images;
calculating a movement distance of the extracted center line between the grouped images, and
Calculating a direction change amount based on the moving distance of the center line
A method of estimating a heading direction further comprising a. In claim 3,
Calculating the moving distance of the extracted center line comprises:
Heading that selects a feature point located on the center line of a reference image from among the grouped images, extracts the feature point from the grouped images, and calculates the number of pixels between the extracted feature point and the center line of each image How to estimate the direction. In claim 2,
Calculating the plurality of reliability ranges comprises:
A method of estimating a heading direction for calculating a reliability range as many as the number of groups based on images corresponding to the same timing for each heading direction value. In claim 1,
When the reference heading direction value is set, calculating a direction change amount from images photographed at different timings, and applying the calculated direction change amount to the reference heading direction value to correct the heading direction value at the current time How to estimate heading direction. In claim 1,
Setting the reference heading direction value includes:
The method of estimating a heading direction further comprising the step of extending the section for the predetermined time or initializing the reference heading direction value when the calculated confidence score is smaller than a set threshold value. In claim 1,
The method of estimating the heading direction further comprising the step of initializing the reference heading direction value when detecting a movement exceeding a threshold value in the lateral direction or the longitudinal direction. A method for a computing device operated by at least one process to correct a heading direction, the method comprising:
Collecting heading direction values from the geomagnetic sensor for a certain period of time, and collecting images from the image sensor;
calculating a plurality of directional changes between the collected images, and calculating a reliability score for each of the heading directional values collected from the geomagnetic sensor based on the directional change;
setting a heading direction value having the highest value among the calculated reliability scores as a reference heading direction value; and
when the reference heading direction value is set, calculating a direction change amount from images taken at different timings, and applying the calculated direction change amount to the reference heading direction value to calculate a heading direction value at the current time
A heading direction correction method comprising a. In claim 9,
Calculating the reliability score comprises:
calculating a plurality of directional changes between the collected images;
calculating a plurality of reliability ranges to which the calculated direction change amount is applied to the heading direction values; and
and calculating a reliability score by counting the cases in which the heading direction value is included in the plurality of reliability ranges. In claim 10,
The step of calculating the reliability score is
If the calculated confidence score is smaller than a set threshold, the heading direction correction method for increasing the section for the predetermined time or initializing the reference heading direction value. In claim 10,
The step of calculating the direction change amount,
If N-1 groupings are performed corresponding to N images, and the movement distance of the extracted center line for each of the grouped images is calculated as the number of pixels of the image, the number of pixels of the image and the size of the pixels of the image are used. to calculate a direction change amount between the grouped images.

Images