KR20240002423A - System for tracking object movement and method thereof - Google Patents

Abstract

An object movement tracking system according to a technical aspect of the present invention includes a first camera provided toward the entrance to acquire a first image including an identification area, and a second camera provided on the ceiling to obtain a second image including a detection area. It may include a camera unit, an object recognition unit that identifies object information from the first image using pre-installed artificial intelligence, and a movement tracker that generates movement information of the object based on the second image. According to an embodiment of the present invention, the moving line of an object can be tracked by generating the moving line information of the object and comparing the moving line information of the object with the pending information when the object disappears or appears in the second image.

Description

Object movement tracking system and method {SYSTEM FOR TRACKING OBJECT MOVEMENT AND METHOD THEREOF}

The present invention relates to an object movement tracking system and method.

In general, online stores collect data on the entire process from customer inflow to repurchase, allowing them to establish marketing strategies and analyze performance, while offline stores must manually collect data other than payment records, so customers It is unknown how people shop within the store.

For example, the customer's behavior within the store, including the length of time spent at each shelf, the store of interest, derivation of products of interest, information on related products, etc., information on the purchase status of products according to the movement path and the customer's movement within the store. As can be seen, the customer's movement within the store is not separately stored or analyzed as data, making it difficult to promote store operation efficiency.

Due to these problems, an increasing number of stores have introduced movement tracking systems. However, the movement tracking system, which uses multiple cameras installed at various angles and directions, is based on the relationship between the image displayed on the actual monitor and the area not captured by the camera. There is a problem that is difficult to understand.

In other words, the entire area must be monitored through images acquired from each camera, but it is not easy to determine the movement of an object using only the images output on the monitor, and the image for the entire area cannot be easily confirmed, so the object is continuously monitored. It is difficult to monitor.

Republic of Korea Patent Publication No. 2001-0016639

The present invention seeks to provide an object movement tracking system and method that recognizes an object using a first image captured toward a doorway and tracks the movement of the object using a second image captured on the ceiling.

One technical aspect of the present invention is a camera unit including a first camera provided toward the entrance to acquire a first image including an identification area and a second camera provided on the ceiling to obtain a second image including a detection area, in advance. It includes an object recognition unit that identifies object information from the first image using artificial intelligence, and a movement line tracking unit that generates movement information of the object based on the second image.

In one embodiment, the movement line tracking unit assigns a new ID to an object that has entered a preset identification area from a preset candidate area in the first image, and the object recognition unit identifies object information to which the ID has been assigned. You can request it.

In one embodiment, the movement line tracking unit extracts the coordinates and feature vectors of the object from the second image at preset intervals and generates movement information including an ID, object information, timestamp, feature vector, and coordinates. .

In one embodiment, the movement line tracking unit recognizes the center of the head area and the center of the body area of the object from the second image, and recognizes the location where the center of the head area is symmetrical with respect to the center of the body area as the foot. The position of the foot can be extracted using the coordinates.

In one embodiment, when an object disappears from the second image, the movement tracking unit manages the movement information of the disappeared object as pending information, and when a new object appears in the second image, it may map the pending information to the new object. there is.

In one embodiment, the movement tracker may determine reserved information to be matched with the new object based on the similarity between the feature vector of the new object and the feature vector of the reserved information.

In one embodiment, the movement tracking unit may determine pending information to be matched with the new object based on the difference between the timestamp of the new object and the timestamp of the pending information.

In one embodiment, the movement tracker may determine pending information to be matched with the new object based on coordinate information of the new object and coordinate information of the pending information.

According to an embodiment of the present invention, the moving line of an object can be tracked by generating the moving line information of the object and comparing the moving line information of the object with the pending information when the object disappears or appears in the second image.

Furthermore, it is possible to determine the customer inflow rate through object movement tracking and analyze the behavior of customers staying in front of specific shelves within the store.

Figure 1 is a block diagram for explaining an object movement tracking system according to an embodiment of the present invention.
FIG. 2 is a block diagram for explaining the object tracking device of FIG. 1.
FIG. 3 is a diagram for explaining the camera unit of FIG. 2.
Figure 4 is a flow chart to explain the process of recognizing an object according to an embodiment of the present invention.
Figures 5a and 5b are exemplary diagrams for explaining steps S400 and S402.
Figure 6 is a flow chart to explain the process of generating movement information of an object.
7A and 7B are exemplary diagrams for explaining steps S600 to S604.
Figure 8 is a flow chart to explain the process of reconnecting an ID whose tracking has been lost according to an embodiment of the present invention.
FIGS. 9A and 9B are exemplary diagrams for explaining FIG. 8.
Figure 10 is a flowchart illustrating a method of tracking the movement of an object according to an embodiment of the present invention.

The advantages and features of the present invention and methods for achieving them will become clear with reference to the embodiments described below along with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below and may be implemented in various different forms. The present embodiments are merely provided to ensure that the disclosure of the present invention is complete and to be understood by those skilled in the art. It is provided to fully inform those who have the scope of the invention, and the present invention is only defined by the scope of the claims.

The terms used in this specification will be briefly explained, and the present invention will be described in detail.

The terms used in the present invention are general terms that are currently widely used as much as possible while considering the function in the present invention, but this may vary depending on the intention or precedent of a person working in the art, the emergence of new technology, etc. In addition, in certain cases, there are terms arbitrarily selected by the applicant, and in this case, the meaning will be described in detail in the description of the relevant invention. Therefore, the terms used in the present invention should be defined based on the meaning of the term and the overall content of the present invention, rather than simply the name of the term.

When it is said that a part "includes" a certain element throughout the specification, this means that, unless specifically stated to the contrary, it does not exclude other elements but may further include other elements. In addition, terms such as “unit”, “module”, “unit”, etc. used in the specification mean a unit that processes at least one function or operation, and include software, hardware components such as FPGA or ASIC, or software and hardware. It can be implemented by combining . However, terms such as “part”, “module”, and “unit” are not limited to software or hardware. A “part”, “module”, “unit”, etc. may be configured to reside on an addressable storage medium and may be configured to reproduce on one or more processors. Accordingly, as an example, terms such as "part", "module", and "unit" refer to components such as software components, object-oriented software components, class components, and task components, and processes. Contains fields, functions, properties, procedures, subroutines, segments of program code, drivers, firmware, microcode, circuits, data, databases, data structures, tables, arrays, and variables. .

Below, with reference to the attached drawings, embodiments of the present invention will be described in detail so that those skilled in the art can easily implement the present invention. In order to clearly explain the present invention in the drawings, parts unrelated to the description are omitted.

Terms containing ordinal numbers, such as “first,” “second,” etc., may be used to describe various components, but the components are not limited by the terms. Terms are used only to distinguish one component from another. For example, a first component may be named a second component, and similarly, the second component may also be named a first component without departing from the scope of the present invention. The term “and/or” includes any one item among a plurality of related items or a combination of a plurality of related items.

Hereinafter, an object movement tracking system according to an embodiment of the present invention will be described with reference to FIGS. 1 to 3.

Figure 1 is a block diagram for explaining an object movement tracking system according to an embodiment of the present invention.

As shown in FIG. 1, the object movement tracking system 1 according to an embodiment of the present invention includes an object tracking device 10 and a movement tracking management server 20.

The object tracking device 10 identifies object information using a first camera provided toward the entrance and a second camera provided on the ceiling, generates movement information of the object, and transmits it to the movement tracking management server 20.

Object information is information for classifying and analyzing incoming and outgoing objects and may include, but is not limited to, the age and gender of the object.

The moving line tracking management server 20 manages the moving line of the object based on the moving line information of the object transmitted from the object tracking device 10.

Movement information may include ID, object information, timestamp, feature vector, and coordinates.

FIG. 2 is a block diagram for explaining the object tracking device of FIG. 1.

As shown in FIG. 2, the object tracking device 10 includes a camera unit 100, a control unit 110, a storage unit 120, and a communication unit 130.

The camera unit 100 includes a first camera 100a that is provided toward the entrance and acquires a first image for identifying objects, and a second camera 100b that is provided on the ceiling and acquires a second image including the detection area. can do.

The first camera 100a acquires a first image including a candidate area and an identification area, and the second camera 100b acquires a second image including a detection area to track the moving line of the object.

The first camera 100a may be configured to capture the face of the object in order to extract information about the object entering and leaving, and the second camera 100b may be configured to capture the entire movement of the object in order to track the movement of the object entering and leaving. It can be provided.

In one embodiment, the first camera 100a captures the entrance from the front and can be implemented as a bullet camera, and the second camera 100b is attached to the ceiling of the store and can capture the entire detection area. It can be implemented with a 3D camera.

In one embodiment of the present invention, a bullet camera and a 3D camera are described, but the type of camera is not limited.

Bullet cameras are small and lightweight cameras formed in the form of a protruding bar rather than a dome. They are used in a wide range of fields due to the increasing demand for crime prevention and order maintenance, and are used not only indoors such as public facilities, subway stations, stores, and schools. , it can be implemented as a device that can be used outdoors, such as in parks, residential alleys, and on the exterior walls of buildings, and can be installed on indoor ceilings or walls.

As such, the first camera 100a is installed facing the entrance of the store and is implemented as a device that can photograph the face of an object entering the identification area from the candidate area.

A 3D camera is a device that can generate 3D coordinates through multiple cameras and obtain 3D coordinate data of the object to be photographed, or an optical sensor that can measure the distance at each point between the 2D camera and the object. It can be implemented as a device that can acquire 3D coordinate data by combining an ultrasonic sensor.

3D cameras can take pictures by rotating the camera lens 360 degrees without installing any tools in front of the camera lens.

As such, the second camera 100b is attached to the ceiling of the store and is implemented as a device capable of photographing objects while rotating 360 degrees.

FIG. 3 is a diagram for explaining the camera unit of FIG. 2.

As shown in FIG. 3, the first camera 100a may be installed so that the front of the store entrance is visible, and a preset candidate area (CA) outside the store entrance and a preset identification area inside the store entrance (Identification area, IA) can be photographed.

In one embodiment of the present invention, the description is based on an object entering the store. In the case of an object leaving the store, the candidate area may be set to the inside of the store entrance, and the identification area may be set to the outside of the store entrance.

In one embodiment of the present invention, by setting a preset candidate area outside the store entrance and a preset identification area inside the store entrance, the object tracking device 10 detects visitors and employees staying around the entrance as objects entering the store. The rate of misrecognition can be reduced.

The second camera 100b may be installed on the ceiling of the store and captures a detection area (DA), which is a space within the store for tracking the movement of objects.

The storage unit 120 stores the movement line information of the object created by the control unit 110.

The storage unit 120 may include at least one of a main memory and an auxiliary memory. For example, the main memory may be implemented using a semiconductor storage medium such as ROM and/or RAM, and the auxiliary memory may be a flash memory device (Solid State Drive (SSD) ), etc.), SD (Secure Digital) cards, hard disk drives (HDD), compact disks, DVDs, or laser disks, etc. can be implemented based on devices that can store data permanently or semi-permanently. .

The communication unit 130 transmits the stored movement route information of the object to the object movement route management server 20.

The communication unit 130 may include a wired or wireless communication network, and the wired communication network may include an Internet network such as a cable network or public switched telephone network (PSTN), and the wireless communication network may include CDMA, WCDMA, GSM, Evolved Packet Core (EPC), It may include LTE (Long Term Evolution), Wibro network, 5G communication network, etc.

The communication unit 130 is not limited to this, and can be used as an access network for a next-generation mobile communication system to be implemented in the future, for example, a cloud computing network in a cloud computing environment, a 5G network, etc., and if the communication unit 130 is a wired communication network, the communication unit 130 The access point within can be connected to the telephone company's exchange office, etc., but in the case of a wireless communication network, data is processed by connecting to the SGSN or GGSN (Gateway GPRS Support Node) operated by the communication company, or BTS (Base Station Transmission), NodeB, or e-NodeB. You can process data by connecting to various repeaters such as.

The control unit 110 may identify object information from the first image and generate movement information of the object from the second image.

In one embodiment, the control unit 110 includes a central processing unit (CPU), a graphics processing unit (GPU), a microcontroller unit (MCU), and an application processor (AP). ), an electronic control unit (ECU), and/or at least one electronic device capable of performing various calculations and control processes. These devices may be implemented, for example, by using one or more semiconductor chips, circuits, or related components alone or in combination.

As shown in FIG. 2, the control unit 110 detects the entry and exit of an object in the first image, and the movement line tracking unit 111 tracks the movement of the object using the second image, and detects the entry and exit of the object from the first image. It includes an object recognition unit 113 that recognizes object information.

Hereinafter, an object recognition method of the control unit in an embodiment of the present invention will be described with reference to FIGS. 4 to 5B.

FIG. 4 is a flowchart for explaining a process for recognizing an object according to an embodiment of the present invention, and FIGS. 5A and 5B are exemplary diagrams for explaining steps S400 to S404.

FIG. 5A is an example diagram showing the first image obtained from the first camera, and FIG. 5B is an enlarged view of the entrance area of FIG. 5A.

As shown in FIG. 5A, the movement line tracking unit 111 detects an object entering a preset candidate area in the first image (S400).

The movement tracker 111 determines whether an object in the first image has entered a preset identification area from a preset candidate area (S402), and assigns a new ID to the object determined to have entered the identification area from the candidate area. Do it (S404).

In one embodiment, the movement line tracking unit 111 sets an identification area for 'Object 1 (Ob 1)' and 'Object 2 (Ob 2)', which are objects located in the candidate area (CA), as shown in FIG. 5B. Determine whether it has entered.

As an example of entry into the identification area (IA), the movement line tracking unit 111 determines that the size of the boundary box among 'Object 1 (Ob 1)' and 'Object 2 (Ob 2)' recognized in the first image is constant. 'Object 2 (Ob 2)', which is smaller than the size, is determined to have not entered the identification area (IA) from the candidate area (CA).

As an example of entry into the identification area (IA), the movement line tracking unit 111 selects 'object 1 (Ob 1)', whose bounding box shown in a square shape in FIG. 5b is larger than a certain size, from the candidate area (CA). It is determined to be an object that has entered the identification area (IA) and a new ID is assigned to 'Object 1 (Ob 1).

In one embodiment, the movement tracking unit 111 does not cross from the candidate area (CA) to the identification area (IA), but keeps moving around in the identification area (IA). It is not determined as a newly entered object, and 'Object 3 (Ob 3) is recognized as a visitor or employee located around the identification area (IA).

In this way, by setting the candidate area and the identification area, the movement tracker 111 can reduce the rate of misrecognition of visitors and employees staying around the store entrance as objects entering the store.

When a new ID is assigned in this way, the movement tracking unit 111 may request the object recognition unit 113 to recognize object information about the object to which the new ID has been assigned.

When there is a request to recognize object information for a plurality of objects, the code handler 115 may add the ID to a queue in the form of a list before passing it to the object recognition unit 113 to identify the object.

The object recognition unit 113, which has received a recognition request from the movement tracking unit 111, identifies object information from the first image using pre-arranged artificial intelligence (S406) and transmits the object information to the movement tracking unit 111. .

Object information may include, but is not limited to, the age and gender of the object.

The object recognition unit 113 can recognize the age and gender of the object extracted from the first image using artificial intelligence.

The object recognition unit 113 can output the age and gender of the object from the object's face using a face recognition algorithm and a face detection algorithm based on artificial neural networks, machine learning, and deep learning.

The object recognition unit 113 sets the face of the object extracted from the first image as input data of the face recognition algorithm, and sets the age and gender of the object as output data of the face recognition algorithm to use the learned face recognition algorithm. there is.

When the object recognition unit 113 completes the identification of object information, the code handler 115 can transmit the next ID to the object recognition unit 113.

The movement line tracking unit 111 tracks the movement line of the object based on the second image (S408).

If object information is not recognized by the object recognition unit 113, the movement line tracking unit 111 does not track the movement line of the object, thereby preventing the collection of unreliable movement line information.

When object information is recognized, the movement line tracking unit 111 analyzes the second image at preset intervals and tracks the object's movement line to generate movement line information.

Hereinafter, a method for generating movement information of an object according to an embodiment of the present invention will be described with reference to FIGS. 6 and 7.

Figure 6 is a flowchart for explaining the process of generating movement information of an object, and Figures 7a and 7b are example diagrams for explaining steps S600 to S604.

Figure 7a is a diagram for explaining the process of extracting coordinates for the foot position.

Figure 7b (a) is an example of a person positioned vertically, Figure 7(b) is an example of a person positioned horizontally, and Figure 7(c) is an example of a person positioned horizontally. is an example of a person with his or her legs covered.

As shown in FIG. 7A, the movement line tracking unit 111 identifies the face of the object from the second image (S600).

The movement tracking unit 111 can be further trained to recognize a person's head through a face recognition algorithm.

When the face of an object is identified, the movement line tracking unit 111 generates a center point (Head Point, HP) for the head.

The movement line tracking unit 111 recognizes the body of the object and creates a center point (Body Point, BP) for the body.

The movement tracking unit 111 recognizes a point symmetrical to the center point of the head based on the center point of the torso as the foot position (Feet Point, FP) (S602).

The movement tracker 111 extracts the location of the recognized foot as coordinates of the foot (S604).

In one embodiment, as shown in Figures 7b (a) to 7b (c), the moving line tracking unit 111 identifies the face of an object from the second image.

The movement tracker 111 uses the center of the torso (yellow) and the position of the head (square box) to determine whether the person is positioned vertically or horizontally, as shown in Figures 7b (a) to 7b (c). You can determine whether it is located in a certain shape.

The movement tracker 111 corrects the position of the center of the body using the aspect ratio of the average body box, as shown in (c) of FIG. 7B.

The movement tracking unit 111 recognizes the position of the foot where the midpoint of the head box is symmetrical with respect to the corrected midpoint of the torso.

The movement tracker 111 extracts the recognized foot position as coordinates.

The second image was captured through a second camera attached to the ceiling, and since it is difficult to accurately determine the position of the foot of the object, accurate coordinates can be obtained by estimating the position of the foot.

The movement line tracking unit 111 extracts a feature vector from the identified face (S606).

The movement line tracking unit 111 can extract a feature vector for the face of an object through an already disclosed feature vector algorithm.

The movement tracker 111 may perform the step of extracting foot position coordinates and the step of extracting feature vectors at the same time or in any order.

The movement line tracking unit 111 generates the movement line information of the object using the foot position coordinates and feature vectors (S608).

The movement tracker 111 extracts the coordinates and feature vectors of the object from the second image at preset intervals and generates movement information including the object's ID, object information, timestamp, feature vector, and coordinates.

If the second image through the second camera does not cover the entire range in which the object can move, there is a blind area where the moving line cannot be detected.

For example, if an object moves through a locker room or staircase where a camera is not installed, tracking of the object's movement is interrupted, and an error in tracking the object's movement may occur.

Accordingly, when an object disappears from the detection area, the movement tracking unit 111 stores the object's movement information as pending information, and matches the stored holding information with an object newly appearing in the detection area, not the identification area, to detect the object stopped by the blind area. The accuracy of movement tracking can be improved.

Hereinafter, a method of reconnecting an ID whose tracking has been lost will be described with reference to FIGS. 8 to 10.

FIG. 8 is a flowchart for explaining a process of reconnecting an ID whose tracking has been lost according to an embodiment of the present invention, and FIGS. 9A and 9B are exemplary diagrams for explaining FIG. 8 .

FIG. 9A is an exemplary diagram for explaining a blind area, and FIG. 9B is an exemplary diagram for explaining a space and a passage.

The movement line tracking unit 111 tracks the movement line of the object through the object's movement line information (S800).

As shown in FIGS. 9A and 9B, the movement line tracking unit 111 determines whether the object has entered a blind area, space, or passage (S802).

In one embodiment of the present invention, blind areas, locker rooms, and passages are defined as areas not captured by cameras, and passages may be spaces such as stairs and hallways.

In one embodiment of the present invention, it is described as a changing room in 9b, but it includes and is not limited to spaces that cannot be photographed with a camera, such as restrooms and lounges.

In one embodiment, it is assumed that the object enters the 'rectangular area' of FIG. 9A.

The movement line tracking unit 111 stores and manages the movement line information of the object that disappeared from the second image as reserved information (S804).

In one embodiment, pending information can be classified and managed by considering the connection relationship between blind areas.

Specifically, in FIG. 9B, the movement line tracking unit 111 may group the rectangular areas by considering the connection relationship between the plurality of rectangular areas.

In one embodiment, as shown in FIG. 9B, the entrance and exit area of the dressing room is set as dead zone 1 (Dead zone, DZ 1), and the two entrance and exit areas for the passage are set as dead zone 2 (Dead zone, DZ 2). It is assumed that it is set by grouping.

The movement tracking unit 111 determines that the object has disappeared by using the rectangular area closest to the last coordinates of the disappeared object, and manages pending information for each rectangular area.

In one embodiment, the movement tracker 111 manages objects that enter blind zone 1 (DZ 1) and disappear from the detection zone (DA) by storing them in a pending information list for blind zone 1 (DZ 1), and Objects that enter Zone 2 (DZ 2) and disappear from the detection zone (DA) are managed by being stored in the pending information list for Rectangular Zone 2 (DZ 2).

The movement line tracking unit 111 detects a new object that has newly entered the detection area in the second image (S806).

In one embodiment, as shown in FIG. 9A, the movement line tracking unit 111 may detect a new object that has newly entered the detection area from the blind area.

In one embodiment, as shown in FIG. 9B, the movement line tracking unit 111 may detect a new object newly entering the detection area at the entrance or exit of blind area 1 (DZ 1) or blind area 2 (DZ 2).

The movement tracking unit 111 extracts a feature vector for the new object and matches the new object with the pending information (S808).

The movement tracker 111 can extract a feature vector for the object's face through a feature vector algorithm.

Movement route information for a new object may not include object information such as gender and age.

The movement tracker 111 may determine pending information to be matched with the new object based on the similarity between the feature vector for the new object and the feature vector in previously stored pending information.

In one embodiment, if the similarity between the feature vector for the new object and the feature vector existing in the reserved information is greater than or equal to the reference value, the movement line tracking unit 111 may match the ID of the new object with the ID of the existing object.

The movement tracking unit 111 can set the standard value of similarity to 90% or more, and the standard value is not limited and can be set differently.

In this way, in FIG. 9B, the movement tracking unit 111 can match the object that newly appears in the detection area in each rectangular area with the pending information stored for each rectangular area.

Meanwhile, in FIG. 9, matching pending information is described based on the feature vector of a new object, but the method of matching a new object and pending information is not limited to this.

The movement tracking unit 111 can further extract a timestamp and coordinates for the new object.

The movement line tracking unit 111 may generate movement information including a timestamp, coordinates, and feature vectors for the new object.

Specifically, in one embodiment, when the similarity between the feature vector of the new object and the feature vector of the previously stored pending information is lower than the reference value, the movement tracker 111 determines the similarity between the timestamp of the new object and the timestamp of the existing object. Based on the difference value, the pending information to be matched with the new object is determined.

In one embodiment, when there is a plurality of pending information in which the similarity between the feature vector of the new object and the feature vector of the previously stored pending information is greater than or equal to the reference value, the movement tracker 111 may store the timestamp of the new object and the timestamp of the existing object. Match new objects and pending information based on the difference value between them.

When the difference value between the timestamp of the new object and the timestamp of the existing object is lower than the reference value, the movement tracking unit 111 recognizes the new object and the existing object as the same person, and sets the ID of the new object to the ID of the existing object. It can be replaced with .

In one embodiment, when the similarity between the feature vector of the new object and the feature vector of the previously stored reserved information is lower than the reference value, the movement line tracking unit 111 creates a new object based on the coordinate information of the new object and the coordinate information of the reserved information. Determine the pending information to be matched with the object.

In one embodiment, when there is a plurality of pieces of high reserved information in which the similarity between the feature vector of the new object and the feature vector of the previously stored reserved information is greater than or equal to the reference value, the movement line tracking unit 111 stores the coordinate information of the new object and the coordinates of the existing object. Match new objects and pending information based on the information.

When the difference between the coordinate information of the new object and the coordinate information of the existing object is smaller than the reference value, the movement tracking unit 111 recognizes the new object and the existing object as the same person and uses the ID of the new object as the ID of the existing object. It can be replaced.

The movement tracker 111 recognizes the new object as the same person as the interval between the timestamp of the new object and the existing object's timestamp is shorter, and the closer the coordinates of the new object are to the existing object, and matches the new object with the existing object. there is.

Hereinafter, a method for tracking the movement of an object according to an embodiment of the present invention will be described using FIG. 10.

Figure 10 is a flowchart illustrating a method for tracking the movement of an object according to an embodiment of the present invention.

The object recognition unit 113 recognizes an object entering the store using the first image (S1000).

The method for recognizing objects has already been described using FIGS. 4 to 5B, so detailed description will be omitted.

The movement line tracking unit 111 uses the second image to generate movement line information of the object recognized in step S1000 and tracks the movement line of the object (S1002).

The movement line tracking unit 111 tracks the movement of an object that enters a blind area, passageway, locker room, etc. and disappears from the detection area and a new object that reappears in the detection area by matching it with a reference object.

The method of generating movement information of an object and tracking the movement of the object has already been described using FIGS. 6 to 9B, and detailed description will be omitted.

The movement line tracking unit 111 transmits the movement line information of the created object to the object movement line management server 20 (S1004).

The object movement route management server 20 can manage the object movement route by storing the object movement route information.

As described above, according to the present invention, the moving line of an object can be tracked by generating the moving line information of the object and comparing the moving line information of the object with the pending information when the object disappears or appears in the second image.

Furthermore, it is possible to determine the customer inflow rate through object movement tracking and analyze the behavior of customers staying in front of specific shelves within the store.

Those skilled in the art related to the embodiments of the present invention will understand that the above-described material may be implemented in a modified form without departing from its essential characteristics. Therefore, the disclosed methods should be considered from an illustrative rather than a restrictive perspective. The scope of the present invention is indicated in the claims, not the detailed description of the invention, and all differences within the equivalent scope should be construed as being included in the scope of the present invention.

10: object estimation device, 100: camera unit,
110: control unit, 111: movement tracking unit,
113: Code handler, 115: Object recognition unit,
120: storage unit, 130: communication unit,
20: object movement management server, 100a: first camera,
100b: second camera

Claims (11)

A camera unit including a first camera provided toward the entrance to acquire a first image including an identification area and a second camera provided on the ceiling to obtain a second image including a detection area;
an object recognition unit that identifies object information from the first image using preset artificial intelligence;
An object movement tracking system including a movement line tracking unit that generates movement information of the object based on the second image.
According to paragraph 1,
The movement line tracking unit,
An object movement tracking system that assigns a new ID to an object that has entered a preset identification area from a preset candidate area in the first image, and requests the object recognition unit to identify object information to which the ID has been assigned.
According to paragraph 1,
The movement line tracking unit,
An object movement tracking system that extracts object coordinates and feature vectors from the second image at preset intervals and generates movement information including ID, object information, timestamp, feature vector, and coordinates.
According to paragraph 3,
The movement line tracking unit,
From the second image, the center of the head area and the center of the body area of the object are recognized, and the location where the center of the head area is symmetrical with respect to the center of the body area is recognized as the foot, and the position of the foot is extracted as the coordinates. Object movement tracking system.
According to paragraph 1,
The movement line tracking unit,
When an object disappears from the second image, an object movement tracking system manages the movement information of the disappeared object as pending information, and maps the pending information to the new object when a new object appears in the second image.
According to clause 5,
The movement line tracking unit,
An object movement tracking system that determines withheld information to be matched with a new object based on the similarity between the feature vector of the new object and the feature vector of the withheld information.
According to clause 5,
The movement line tracking unit,
An object movement tracking system that determines pending information to be matched with a new object based on the difference between the timestamp of the new object and the timestamp of the pending information.
According to clause 5,
The movement line tracking unit,
An object movement tracking system that determines pending information to be matched with a new object based on the coordinate information of the new object and the coordinate information of the pending information.
In an object movement tracking method performed using an object movement tracking system,
Obtaining a first image including an identification area using a first camera provided toward the entrance;
Obtaining a second image including the detection area using a second camera provided on the ceiling;
Identifying object information from the first image using previously prepared artificial intelligence; and
A step of generating movement information of the object based on the second image,
The step of identifying object information from the first image includes:
An object movement tracking method that assigns a new ID to an object that has entered a preset identification area from a preset candidate area in the first image, and requests the object recognition unit to identify object information to which the ID has been assigned.
According to clause 9,
The step of generating movement information of the object is,
At each preset period, the coordinates and feature vectors of the object are extracted from the second image, and movement information including ID, object information, timestamp, feature vector, and coordinates is generated,
From the second image, the center of the head area and the center of the body area of the object are recognized, and the location where the center of the head area is symmetrical with respect to the center of the body area is recognized as the foot, and the position of the foot is extracted as the coordinates. Object movement tracking method.
According to clause 9,
When an object disappears from the second image, managing the movement information of the disappeared object as pending information, and mapping the pending information to the new object when a new object appears in the second image.