KR20220155700A - People counting method and apparatus using artificial intelligence - Google Patents

Abstract

The present invention relates to a people counting method and device using artificial intelligence. More specifically, the present invention relates to the people counting method and device using artificial intelligence, which quickly and accurately detects the number of people by using a pattern of a part where people are located. According to the present invention, the people counting method using artificial intelligence may comprise: a photographing step in which a camera module photographs an image of surroundings; and a detection step of detecting the number of objects included in the target image by analyzing a target image photographed by the camera module.

Description

Crowd counting device and method using artificial intelligence {PEOPLE COUNTING METHOD AND APPARATUS USING ARTIFICIAL INTELLIGENCE}

The present invention relates to a crowd counting device and method using artificial intelligence. More specifically, the present invention relates to a crowd counting apparatus and method using artificial intelligence for quickly and accurately detecting the number of crowds using a pattern of a part where the crowd is located.

Recently, not only music concerts or sporting events at stadiums, but also various crowd rallies and gatherings in specific places are increasing.

Media organizations that host the gathering or collect and analyze information about the gathering need to accurately determine the number of people gathered at the rally.

Conventionally, a statistical method of inferring the total number of people by directly counting manpower or by extracting a sample from a crowded crowd and counting the number of people in the sample was used.

This method has a problem in that a considerable amount of labor cost is required when using manpower and an excessively long time is required to detect the number of the crowd.

As a prior art for solving these problems, Korean Patent Laid-open Publication No. 10-2013-0065234 (title of invention: object automatic counting method, 2013.06.19.) (Document 1) discloses pixels of a specific color of an object in a captured image. A technique for detecting the number of objects by extracting and counting the number of extracted pixels is disclosed.

However, the technique according to Document 1 has a problem in that it is difficult to accurately detect the number of objects when the objects do not have a specific color.

In addition, in the technique according to Document 1, since each pixel of a specific color is counted, it takes a considerable amount of time to detect the number of objects, and the calculation is complicated.

[Document 1] Republic of Korea Patent Publication No. 10-2013-0065234

An object of the present invention is to provide a crowd counting device and method using artificial intelligence for detecting the number of crowds by using a pattern of a part where the crowd is located.

In the crowd counting method using artificial intelligence according to the present invention, a camera module captures a surrounding image and analyzes the target image captured by the camera module to determine the number of objects included in the target image. A detection step of detecting the number may be included.

In addition, the camera module may include a first camera part and a second camera part, and at least a portion of a photographing area of the first camera part and the second camera part may be different from each other.

In addition, the step of determining and storing positional information on the photographing area of the camera module, the step of determining and storing information on the installation position of the camera module, and the step of determining at least one object in the target image are further performed. In the detecting step, the number of the objects may be detected based on location information of the object in the target image and reference information of the object at the location of the object.

The camera module may further include taking and storing a comparison image of the surroundings and detecting the number of objects in the comparison image, and in the detecting step, the target image is converted into at least one of the objects. The number of objects may be detected by comparing with a comparison image.

Crowd counting device using artificial intelligence according to the present invention detects the number of objects included in the target image by analyzing a camera module for capturing a surrounding image and a target image captured by the camera module. It may include a detection unit that does.

The crowd counting apparatus and method using artificial intelligence according to the present invention has an effect of enabling rapid crowd counting because the number of crowds is detected using a pattern of a part where crowds gather.

1 to 3 are views for explaining a crowd counting device using artificial intelligence according to the present invention.
4 to 7 are diagrams for explaining a method of setting and using reference information.
8 to 20 are diagrams for explaining a comparison image and a reference image.
21 to 23 are diagrams for explaining the canonical coefficient mode.
24 to 29 are views for explaining a camera module.
30 to 32 are diagrams for explaining a case in which a camera module is installed in an electric train.

Hereinafter, a crowd counting device and method using artificial intelligence according to the present invention will be described in detail with reference to the accompanying drawings.

Since the present invention can make various changes and have various embodiments, specific embodiments will be illustrated in the drawings and described in detail in the detailed description. This is not intended to limit the present invention to specific embodiments, and it can be understood to include all changes, equivalents, and substitutes included in the spirit and technical scope of the present invention.

In describing the present invention, terms such as first and second may be used to describe various components, but the components may not be limited by the terms. The terms may only be used for the purpose of distinguishing one component from another. For example, a first element may be termed a second element, and similarly, a second element may be termed a first element, without departing from the scope of the present invention.

The terms and/or may include any combination of a plurality of related recited items or any of a plurality of related recited items.

When a component is referred to as being "connected" or "connected" to another component, it may be directly connected or connected to the other component, but other components may exist in the middle. can be understood On the other hand, when an element is referred to as “directly connected” or “directly connected” to another element, it may be understood that no other element exists in the middle.

Terms used in this document are only used to describe specific embodiments and are not intended to limit the present invention. Singular expressions may include plural expressions unless the context clearly dictates otherwise.

In this document, the terms "include" or "having" are intended to indicate that there is a feature, number, step, operation, component, part, or combination thereof described in the specification, and one or more other features It may be understood that the presence or addition of numbers, steps, operations, components, parts, or combinations thereof is not precluded.

Unless defined otherwise, all terms used herein, including technical or scientific terms, may have the same meaning as commonly understood by a person of ordinary skill in the art to which the present invention belongs. Terms such as those defined in commonly used dictionaries may be interpreted as having a meaning consistent with the meaning in the context of the related art, and unless explicitly defined in the present application, interpreted in an ideal or excessively formal meaning. It may not be.

In addition, the embodiments disclosed in this document are provided to more completely explain the embodiments to those skilled in the art, and the shapes and sizes of elements in the drawings may be exaggerated for clearer explanation.

In describing the present invention in this document, if it is determined that a detailed description of a related known function or configuration may unnecessarily obscure the gist of the present invention, the detailed description may be omitted.

Various embodiments described in this document may be implemented in a recording medium readable by a computer or similar device using software, hardware, or a combination thereof.

According to hardware implementation, embodiments of the present invention are ASICs (application specific integrated circuits), DSPs (digital signal processors), DSPDs (digital signal processing devices), PLDs (programmable logic devices), FPGAs (field programmable gate arrays, processors It may be implemented using at least one of processors, controllers, micro-controllers, microprocessors, and electrical units for performing functions.

Meanwhile, according to software implementation, embodiments such as procedures or functions in the present invention may be implemented together with a separate software module that performs at least one function or operation.

1 to 3 are views for explaining a crowd counting device using artificial intelligence according to the present invention.

1, a crowd counting device (1, hereinafter referred to as "counting device") using artificial intelligence according to the present invention may include a camera module (Camera Module, 10) and a server (Server, 20) have.

The camera module 10 may capture images of the surroundings. Here, the periphery may mean an area where the camera module 10 can take pictures, that is, a picture area.

The server 20 may perform communication with at least one camera module 10 in a wired and/or wireless manner.

The server 20 may receive an image captured by the camera module 10 from at least one camera module 10 and analyze the transmitted image.

The server 20 may determine the number of objects OB in the image captured by the camera module 10 using artificial intelligence.

In addition, the server 20 may manage at least one camera module 10 .

The image captured by the camera module 10 may include a target image, a comparison image, and a sample image.

The sample image may also be referred to as a partial image.

The target image may be an image captured to detect the number of crowds. For example, in the case of determining the number of crowds gathered at AA Plaza at aa: xx/xx/xx/20xx, the camera module 10 captures AA Plaza at aa: xx/xx/xx/20xx at aa: the target image. It can be.

The comparison image may be an image used for comparison with a target image for the purpose of quickly and accurately detecting the number of crowds in the target image.

The comparison image may include a comparison pattern block.

The comparison pattern block is a type of image block used for comparison with a target image for the purpose of quickly and accurately detecting the number of crowds in the target image. Hereinafter, the comparison pattern block may be referred to as a comparison block.

The server 20 may periodically or non-periodically add, change, or delete at least one comparison image (or comparison block).

The sample image may be an image used to generate a reference image.

The target image, comparison image, sample image, and comparison block will be more clearly described through the following description.

Referring to FIG. 2 , the camera module 10 includes a first control unit 100, a first communication unit 110, a first interface unit 120, a memory unit 130, a camera location information unit 140, and a first camera unit. (11), a first adjusting unit 160, a second camera unit 12, and a second adjusting unit 180 may be included.

Since the components shown in FIG. 2 are not essential, it is also possible to implement the camera module 10 having more components or fewer components.

The reason for using the term “first” in relation to the first control unit 100, the first communication unit 110, and the first interface unit 120 is to distinguish them from the components of the server 20 to be described later. .

For example, the reason why the term "first" is used in the first communication unit 110 is to distinguish it from the communication unit 210 of the server 20, that is, the second communication unit 210, which will be described later.

The first controller 100 may control overall functions and operations of the camera module 10 . For example, the first controller 100 may control image capture of the camera module 10 and may control communication with the server 20 .

The first communication unit 110 may perform communication with other devices, for example, the server 20, in a wired and/or wireless manner under the control of the first control unit 100.

The first interface unit 120 may provide a passage for connection with other devices under the control of the first controller 100 .

The memory unit 130 may store various programs, data, etc. required for operation and/or function implementation of the camera module 10 under the control of the first controller 100 .

The camera location information unit 140 may determine location information of the camera module 10 under the control of the first controller 100 . For example, the camera location information unit 140 may determine GPS coordinate information of the camera module 10 .

When the camera module 10 is installed inside a vehicle or building, the camera location information unit 140 may determine information about the location of the camera module 10 within the vehicle or building. For example, when the camera module 10 is installed in a train used in a subway or the like, the camera location information unit 140 includes unique information such as a unique number of the train in which the camera module 10 is installed, and the camera module in the train. Information about the location of (10) can be determined.

The first camera unit 11 may capture an image of the surroundings under the control of the first controller 100 . For example, the first camera unit 11 may capture an image of the surroundings in a still image type such as a video and/or a photo.

The first adjusting unit 160 may adjust the photographing angle by rotating or swinging the first camera unit 11 under the control of the first controller 100 .

The second camera unit 12 may capture an image of the surroundings under the control of the first controller 100 . For example, the second camera unit 12 may capture an image of the surroundings in a still image type such as a video and/or a photo.

The second adjusting unit 180 may adjust the photographing angle by rotating or swinging the second camera unit 12 under the control of the first controller 100 .

Referring to FIG. 3 , the server 20 includes a second control unit 200, a second communication unit 210, a second interface unit 220, a database unit (DB, 230), a photographing area location information unit 240, a standard It may include an information unit 250, a comparison image unit 260, a reference image unit 270, and a detection unit 280.

Since the components shown in FIG. 3 are not essential, it is also possible to implement the server 20 with more or fewer components.

The second controller 200 may control overall functions and operations of the server 20 . For example, the second controller 200 may control settings for crowd counting and may control a crowd counting process.

The second communication unit 210 may perform communication with other devices, for example, the camera module 10, in a wired and/or wireless manner under the control of the second control unit 200.

The second interface unit 220 may provide a passage for connection with other devices under the control of the second control unit 200 .

The database unit (DB, 230) may store various programs, data, etc. required for operation of the server 20 under the control of the second control unit 200.

For example, the database unit (DB, 230) provides information on the installation position of each camera module 10 (location information of the camera module 10) and an area (photographing area) corresponding to the camera module 10. information can be stored and managed.

The database unit (DB, 230) may store and manage preset reference information, comparison images, and reference images.

In addition, the database unit (DB, 230) may store and manage information on various preset image patterns, comparison blocks, and the like.

The capturing area location information unit 240 may determine location information of an area corresponding to the camera module 10 , that is, a capturing area, under the control of the second controller 200 .

Here, the capturing area may be referred to as an area included in an image captured by the camera module 10 . Alternatively, the capturing area may also be referred to as a capturing area of the camera module 10 .

The standard information unit 250 may generate,/or determine, and manage standard information, that is, standard information, to determine an object under the control of the second controller 200 .

Here, the object may mean a person included in the crowd. For example, the objects may include men, women, adults, children, infants, the disabled, the elderly, and the like.

The reference information may include information about the shape of the object, information about the size (key, etc.) according to the location of the object, and the like.

Reference information will be described in more detail below.

The comparison image unit 260 may create, store, and manage comparison images and/or comparison blocks under the control of the second controller 200 .

The comparison image may be an image compared with a target image to determine the number of crowds.

The comparison block is an image compared with a target block to determine the number of crowds, and may be a part of the comparison image.

The comparison image unit 260 may include a color information unit 261 and a shape information unit 262 .

The color information unit 261 may generate, store, and manage color information of the comparison image.

For example, the color of the image may vary depending on the time, season, weather, etc. at which the camera module 10 captures the comparison image.

The color information unit 261 may match variables such as time, season, and weather at which the comparison image is taken and information on color.

The shape information unit 262 may create, store, and manage information on various shapes of crowd groups and/or objects included in the comparison image.

For example, the shape of an object or crowd group may vary depending on an angle at which the camera module 10 captures a comparison image, a pose of the object, clothing such as a hat, and a type of crowd gathering.

The shape information unit 262 may create, store, and manage information about the shape of the object and/or crowd group.

The comparison image will be described in more detail below.

The detection unit 280 analyzes the target image captured by the camera module 10 under the control of the second controller 200 or compares the target image with the comparison image, and the number of objects included in the target image (of the crowd) number) can be detected.

The detection unit 280 may include an object detection unit 281, an object number detection unit 282, and a summing unit 283.

The object detection unit 281 may detect an object (person) by using standard information on size and standard information on shape in an image captured by the camera module 10 .

The object count detection unit 282 may detect the number of objects in the image captured by the camera module 10 . To this end, the object count detector 282 may use information detected by the object detector 281 or a comparison image and a target image.

The summing unit 283 may detect the total sum of the number of objects (the total number of crowds) in the target image captured by the camera module 10 based on the information detected by the object count detection unit 282 .

A method of detecting the number of crowds will become clearer through the following description.

4 to 7 are diagrams for explaining a method of setting and using reference information. Hereinafter, description of the parts described in detail above may be omitted.

As shown in FIG. 4 , in the setting mode, the camera module 10 may capture an image of the surroundings (S100).

Here, the setting mode may be referred to as a mode in which reference information, comparison images, and/or reference images are generated and prepared for crowd counting.

An image captured by the camera module 10 may be transmitted to the server 20 through the first communication unit 110 .

Then, the reference information unit 250 of the server 20 may analyze the image received from the camera module 10 according to the control of the second control unit 200 (S200).

Based on the analysis result, the reference information unit 250 may determine location information of the imaging area (S210).

Location information of the capturing area may include GPS coordinate information.

Alternatively, the location information of the capturing area may include relative location information with respect to the camera module 10 .

For example, how far is the first point of the photographing area from the camera module 10, what is the photographing angle of the camera module 10 corresponding to the first point, and the camera module 10 is the reference point. Information about the direction in which one point is located may be used as location information of the imaging area.

In addition, the server 20 may determine the location information of the camera module 10 and the location information of the point where the side camera module 10 is installed (S220).

Here, the location information of the camera module 10 may include GPS coordinate information of the camera module 10 and location information in a vertical direction of the camera module 10, that is, altitude information (height information). .

Thereafter, the reference information unit 250 may determine standard size information according to a location on the imaging area by using location information of the imaging area and location information of the camera module 10 (S230).

Assume that the camera module 10 captures an area (photographing area) as shown in FIG. 5 .

The reference information unit 250 determines a first point P1, a second point P2, a third point P3, and a fourth point (P1), a second point (P2), and a fourth point ( The location information of P4) can be determined.

Here, a case of determining and using location information for a total of four points in a photographing area is described for better understanding, but the present invention may not be limited thereto. For example, in the present invention, it is possible to determine and use location information for all points in the imaging area.

The location information includes GPS coordinate information of a corresponding point, information about a distance from the camera module 10, information about a photographing angle of the camera module 10 corresponding to the corresponding point, and a direction relative to the camera module 10. information may be included.

The reference information unit 250 may generate and determine reference information according to the location based on the determined location information of the capturing area.

Let's assume that the height of an average person is 50 cm (such as infants) to 200 cm (such as adults).

In this case, reference information about the height of the object (person) corresponding to the second point P2 may be set to [10m, 0.5cm~2.0cm, 50cm~200cm].

For example, as shown in FIG. 6, in the image taken by the camera module 10, the height of a person located at the second point P2 10 m away from the camera module 10 (Pc) is 0.5 cm to 2.0 cm. If it is expressed as cm (L1), it may mean that the person's actual height is 100 times 50 cm to 200 cm.

The reference information on the height of the object (person) corresponding to the first point P1 farther from the camera module 10 than the second point P2 is [50m, 0.1cm~0.4cm, 50cm~200cm]. it is possible to set

For example, as shown in FIG. 6, in the image taken by the camera module 10, the height of a person located at a point P1 50 m away from the camera module 10 (Pc) is 0.1 cm to 0.4 cm ( L2), it may mean that the actual height of the person is 50 cm to 200 cm, which is 500 times.

Assume that an object with a height of 1.8 cm is detected at a position P2 10 m away from the camera module 10 in an image captured by the camera module 10 .

In this case, if the reference information on the height of the object mentioned above is applied, the height of the object is approximately 180 cm, and accordingly, it can be determined that the height of the object corresponds to a normal person.

Of course, in the case of an unusually tall person such as a sufficiently short infant or basketball player, it is possible to apply other reference information about height.

Then, it may be possible to determine that the detected object is an object (person) based on the shape of the detected object.

As shown in FIG. 7 , suppose an object having an excessively tall height, for example, 1.8 cm, is located at the first point P1 in the image captured by the camera module 10 .

In this case, the actual height of the object determined based on the reference information may be approximately 9 m.

Since it is sufficiently rare or impossible for a person to have a height of 9m, the object can be judged as a statue, mannequin, signboard, mural, or video rather than an object (person).

In this case, the object may not be counted.

In order to generate and/or determine reference information, it is possible to consider not only the distance from the camera module 10 but also the shooting angle according to the position in the shooting area.

For example, when the camera module 10 is installed in a high-rise building or the like, a photographing angle may be different depending on a location on a photographed image.

Reference information may include object shape information.

For example, in the case of a normal person, it is possible to have a long shape with a head, arms, legs and torso. Such information on the shape of a normal person may be used as reference information.

If an object supporting the ground with four feet and having a tail appears in the image captured by the camera module 10, the object may be an animal such as a dog or horse rather than an object (human) is high

Therefore, in this case, it may not be counted.

As described above, when the standard information unit 250 generates and/or determines reference information according to the location of the capturing area, it is possible to quickly and precisely determine the number of objects regardless of the environment of the captured image. can

For example, let's assume a case where the same criterion is set corresponding to all the camera modules 10 without setting the criterion information corresponding to the photographing area.

In this case, the possibility of an error occurring in the process of detecting an object or the number of objects may be relatively high due to environmental factors such as the curvature or slope of the floor of the imaging area and the arrangement of surrounding buildings.

On the other hand, when the camera module 10 determines and generates reference information corresponding to a capturing area, the reference information is generated in consideration of surrounding environmental factors of the corresponding capturing area, so that a more precise and rapid object or object can be detected. Detection of numbers may be possible.

The server 20 may periodically or non-periodically update, update, add, or delete the reference information.

For example, when the server 20 analyzes an image captured by the camera module 10 and determines that a part other than an object, for example, a part that becomes a background such as a building or a road, is changed, new reference information is generated. and/or can be identified.

In detail, when changing the reference image, it may be possible to newly generate and/or determine and change the reference information as well.

A method of changing the reference image will be described in detail with reference to FIG. 19 below.

The reference information unit 250 of the server 20 may determine an object (person) from an image captured by the camera module 10 using the reference information described above.

Hereinafter, a comparison image and a reference image will be described with reference to the accompanying drawings.

8 to 20 are diagrams for explaining a comparison image and a reference image. Hereinafter, description of the parts described in detail above may be omitted.

Referring to FIG. 8 , when the camera module 10 captures a surrounding image in the setting mode (S100), the comparison image unit 260 may determine whether to set the captured image as a comparison image (S110).

As a result of the determination in step S110, when the captured image is not set as a comparison image, a preset first function (Default 1) may be performed (S120).

The first function may include, for example, a function of determining whether to set the next image as a comparison image, a function of determining whether to end setting of a comparison region, and the like.

On the other hand, as a result of the determination in step S110, when the captured image is set as a comparison image, the comparison image unit 260 may detect the number of objects in the corresponding image (S130).

Thereafter, the comparison image unit 260 may store the corresponding image as a comparison image (S140).

Meanwhile, it is possible to set the reference image after the camera module 10 captures the surrounding image (S100).

Contents of the reference image will be described in more detail after FIG. 17 .

Assume that the images shown in (A) and (B) of FIG. 9 are set as comparison images.

In the case of (A) of FIG. 9 , it can be said that the objects OB are scattered singly within the photographing area in the image captured by the camera module 10 .

In this case, the number of objects included in the image, that is, the number of crowds, may be detected based on preset reference information.

For example, in the image of (A) of FIG. 9, the location information of the object OB (the location information of the point where the object OB is located) and the key, shape, etc. of the object OB at the location of the object OB The number of objects can be detected based on the reference information of This can be performed by the object detection unit 281 and the object number detection unit 282 of the detection unit 280 of the server 20 using reference information.

Thereafter, the pattern of the image of FIG. 9(A) may be stored, and information on the number of objects included in the image of FIG. 9(A) may be matched to the corresponding pattern and stored.

For example, as shown in FIG. 10 , an image corresponding to the pattern of FIG. 9 (A) may be set as a first comparison image and stored.

In addition, information on the number of objects (number of people) included in the first comparison image, the time at which the first comparison image was taken, and the weather or field of view corresponding to the first comparison image may be matched and stored with the first comparison image. .

In the case of (B) of FIG. 9 , it can be said that the objects OB in the image captured by the camera module 10 are gathered in groups at a predetermined point within the capturing area.

Even in this case, the number of objects included in the image, that is, the number of crowds, can be detected based on preset reference information.

Thereafter, the pattern of the image of FIG. 9 (B) may be stored, and information on the number of objects included in the image of FIG. 9 (B) may be matched to the corresponding pattern and stored.

For example, as shown in FIG. 10 , an image corresponding to the pattern of FIG. 9 (B) may be set as a second comparison image and stored.

In addition, information on the number of objects (number of people) included in the second comparison image, the time at which the second comparison image was taken, and the weather or field of view corresponding to the second comparison image may be matched and stored with the second comparison image. .

The comparison image unit 260 may periodically and/or non-periodically add or update comparison images.

As the number of comparison images increases, precise crowd counting may be possible, and thus the comparison image unit 260 may add new comparison images at preset intervals.

If it is sufficiently similar to the comparison image to be added (if the similarity is greater than a preset threshold similarity), it is possible to delete the previous image.

Assume that the comparison image unit 260 generates a new 10th comparison image while the first comparison image is stored.

Here, when the similarity between the first comparison image and the tenth comparison image is greater than a preset threshold similarity, the first comparison image may be deleted, and the tenth comparison image may be newly stored.

Of course, it may be possible to store the 1st comparison image and the 10th comparison image having a sufficiently high degree of similarity together for more accurate crowd counting.

The fact that the number of comparison images is large (the number of comparison blocks is large) may indicate that the learning of the artificial intelligence of the server 20 is more intensified.

In addition, as the number of comparison images (the number of comparison blocks) increases, more precise crowd counting may be possible.

It may be possible to periodically and/or non-periodically create, update, and add a comparison area (comparison block) to deepen the learning of the artificial intelligence of the server 20 .

Assume that the camera module 10 captures a predetermined first image in the crowd counting mode. Here, the crowd counting mode may be referred to as a mode for detecting the number of objects included in an image captured by the camera module 10 . This crowd counting mode will be described in detail later in FIG. 21 .

When the camera module 10 captures the first image and transmits it to the server 20, the comparison image unit 260 of the server 20 may analyze a pattern of the received first image.

In addition, the comparison image unit 260 may compare the pattern of the first image with pre-stored comparison images to determine a similar image pattern.

For example, when it is determined that the pattern of the first image is most similar to the first comparison image stored in advance, the comparison image unit 260 determines that the first comparison image is most similar to the pattern of the first image, and the detection unit 280 ) can be transmitted.

Then, the object number detection unit 282 of the detection unit 280 may determine information about the number of objects included in the first comparison image, and detect the determined information as the number of objects included in the first image.

In the above, the method of setting the image taken by the camera module 10 as a comparison image as a whole has been described, but the present invention may not be limited thereto.

For example, a partial area including an object OB is extracted from an image captured by the camera module 10, that is, a comparison image, and a pattern of the image corresponding to the object OB in the extracted partial area is set as a comparison block. It is possible.

Here, the comparison block may also be referred to as a comparison image of another type.

From this point of view, the comparison block can be said to be an image having a relatively small size compared to the comparison images described above in (A) and (B) of FIG. 9 . Considering this, the term block can be used.

Assume that the camera module 10 captures an image as shown in (A) of FIG. 11 and the comparison image unit 260 sets the captured image as a comparison image.

The comparison imaging unit 260 may set the region where the object OB is located in the image of FIG. 11 (A), that is, the first region AR1 and the second region AR2, respectively, as regions of interest.

Then, as shown in (B) of FIG. 11 , it is possible to set image blocks to surround the periphery of the objects OB included in the region of interest.

For example, it is possible to set an image block to surround the objects OB included in the first area AR1, set the set image block as the first comparison block PT1, and store the set image block. In other words, it is possible to set a part of the comparison image as the first comparison block PT1.

After setting the first comparison block PT1 in the first area AR1, the number of objects included in the first comparison block PT1 may be detected.

For example, the detection unit 280 of the server 20 may detect the number of objects included in the first comparison block PT1.

A plurality of objects OB may overlap and be positioned in the first comparison block PT1. In this case, the detector 280 may detect the number of objects included in the first comparison block PT1 in a different way. This will be described in detail later.

Here, the comparison block may correspond to a part where one or more objects OB are located.

From another point of view, the comparison block may mean a pattern of an image corresponding to a single object or a plurality of objects OB.

Assume that a plurality of objects OB are gathered at a predetermined location.

In this case, it may be faster to detect the number of objects by analyzing a pattern in which a plurality of objects OB are gathered rather than counting the number of objects OB one by one.

As the number of comparison blocks increases, the accuracy of the crowding coefficient can be improved.

The comparison image unit 260 may store pattern information of the comparison block, match information on the number of objects (number of people) corresponding to the comparison block, and store it in the database (DB, 230) of the server 20. have.

When the second area AR2 of FIG. 11(A) is compared with the first area AR1, the size of the second area AR2 is relatively small, but the first comparison block PT1 is also included in the second area AR2. ) may be included. In addition, the distance D2 between the camera module 10 and the second area AR2 may be greater than the distance D1 between the camera module 10 and the first area AR1.

In the image captured by the camera module 10, the size of the second area AR2 is smaller than that of the first area AR1, but the number of objects included in the first area AR1 and the size of the second area AR2 It may indicate that the number of objects included in may be the same or similar.

Considering this, the information on the comparison block is information about the pattern of the comparison block, information about the number of objects included in the comparison block, information about the size of the comparison block, and the distance between the camera module 10 and the comparison block. information, information on a direction in which the comparison block is located with respect to the camera module 10, and the like.

Information on these comparison blocks may be matched with each other and stored in the database 230 of the server 20 .

Comparing the first area AR1 and the third area AR3 of FIG. 12 , the sizes of the first area AR1 and the third area AR3 are similar to each other, and the first area AR1 and the third area ( AR3) may all include the first comparison block PT1. In addition, the distance D3 between the camera module 10 and the third area AR3 may be greater than the distance D1 between the camera module 10 and the first area AR1.

Considering this, the size of the third area AR3 in the image captured by the camera module 10 is similar to or equal to the size of the first area AR1, but the number of objects included in the third area AR3 is It may indicate that the number of objects included in the first area AR1 may be greater than the number of objects included in the first area AR1.

Considering this, the pattern shape of the comparison block and the distance from the camera module 10 can also be said to be important variables for crowd counting.

13 shows information on various comparison blocks.

Referring to FIG. 13, in relation to the first comparison block PT1, the basic number of objects (basic number of people) is 5, the weight according to the distance to the camera module 10 (distance weight) is 1.2 per meter, and the width weight is It may be 1.02 per square meter.

The distance weight may indicate that the weight increases by 1.2 whenever the distance between the first comparison block PT1 and the camera module 10 increases by a unit distance (eg, 5m) from a preset basic distance (eg, 10m). .

The possible area may indicate that the weight is increased by 1.02 whenever the area of the first comparison block PT1 increases by the unit area (eg, 1 square meter) from the basic area (eg, 2 square meters).

The number of basic objects of the first comparison block PT1 is determined when the area of the first comparison block PT1 has the basic area and the distance between the first comparison block PT1 and the camera module 10 is the basic distance. The number of objects included in the first comparison block PT1 may be indicated.

For example, when the area of the first comparison block PT1 is the basic area (eg, 2 square meters) and the distance between the first comparison block PT1 and the camera module 10 is the basic distance (eg, 10 m), The number of objects included in the first comparison block PT1 may be 5 corresponding to the basic number of objects.

If the area of the first comparison block PT1 is wider than the basic area by a unit area (for example, 3 square meters), and the distance between the first comparison block PT1 and the camera module 10 is larger than the basic distance by a unit distance, If it is large (for example, 15 m), the number of objects included in the first comparison block PT1 may be 6.12 (5 × 1.2 × 1.02).

Here, the case where the distance weight is set to 1.2 and the width weight is set to 1.02 has been described, but these are arbitrarily set for easy understanding, and the present invention may not be limited thereto.

In this way, the pattern shape information of the comparison block, information on the number of objects included in the comparison block, information on the distance of the comparison block to the camera module 10, and information on the size (width) of the comparison block are matched. In the case of storage, quick and precise crowd counting is performed by finding similar comparison blocks and checking information on the number of objects corresponding to them without counting objects OB in the image captured by the camera module 10 one by one. can be done

Meanwhile, the comparison block may have various forms.

For example, as shown in (A) and (B) of FIG. 14 , the second comparison block PT2 may have a donut shape with an empty center.

When a plurality of objects OB gather to play Ganggangsullae, a second comparison block PT2 of the type shown in (A) of FIG. 14 may be formed.

In the case of (B) of FIG. 14, the second comparison block PT2 is formed identically to that of (A), but the number of objects OB included in the comparison block may be relatively greater.

When a plurality of objects OB have a campfire or the like, a second comparison block PT2 of the type shown in (B) of FIG. 14 may be formed.

The difference in the number of objects in (A) and (B) of FIG. 14 can be determined using the above-described distance weight and/or area weight.

In this way, information on the shape of the pattern, information on the number of objects included in the comparison block, information on distance, information on area, etc. are matched in response to the comparison block including at least one object OB. When stored, crowd counting can be performed more quickly and precisely.

On the other hand, when a plurality of objects (OB) overlap each other in the comparison image or comparison block, it may be difficult to accurately determine the number of objects.

In this case, it is possible to determine the number of objects sufficiently accurately by replaying previously stored video type images in reverse. A comparison block having an image pattern in which a plurality of objects OB overlap may be referred to as an overlapping block.

As shown in FIG. 15, in the process of detecting the number of objects (S130), it may first be determined whether or not a plurality of objects OB overlap a comparison image or a comparison block (S131).

As a result of determination in step S131, when the plurality of objects OB do not overlap, the object detection unit 281 and the number of objects detection unit 282 of the detection unit 280 count the number of objects included in the comparison image or comparison block. It can be determined by doing (S134).

On the other hand, as a result of the determination in step S131, when a plurality of objects OB overlap, the detection unit 280 may reversely reproduce the previously stored image (S132).

To this end, the server 20 can store the image captured by the camera module 10 in the database (DB, 230) in the form of a video.

Let's assume a case where a pattern of a part of an image at a point of aaa is stored as a comparison block in an image captured by the camera module 10 . A pattern of a part of the image at the aaa point of view may be referred to as an overlapping block, which is a kind of comparison block.

In this case, the detection unit 280 may reversely reproduce the image captured by the camera module 10 from the point aaa.

In addition, the number of objects may be determined by tracking the movement of each object OB included in the corresponding comparison block (S133) and counting the number of tracked objects OB while replaying the video in reverse.

For example, assume a case where the comparison image unit 260 stores a pattern having a shape as shown in (A) of FIG. 16 as a third comparison block PT3.

The comparison block of FIG. 16 (A) can be viewed as having an image pattern in which a plurality of objects OB overlap. 16(A) can be regarded as a kind of overlapping block.

In this case, the detection unit 280 may reversely reproduce a video type image captured by the camera module 10 .

Then, as shown in (B) and (C) of FIG. 16, each object OB moved or separated from the group (third comparison block PT3) in the process of reproducing the image taken by the camera module 10 backwards. It is possible to track the movement (movement, movement) of the objects and count the number of objects (OB).

When the movement of each object OB is tracked, a change in image data occurs, and the number of objects OB can be detected by detecting the change in image data.

If this method is used, the number of objects can be accurately determined even in a comparison block (third comparison block PT3) in which a large number of objects OB are gathered like overlapping.

The server 20 stores information such as pattern shape and size of the third comparison block PT3 and information on the number of objects OB corresponding to the third comparison block PT3 to the database (DB, 230). can be saved

Meanwhile, in order to easily detect the object OB in an image captured by the camera module 10, it is possible to set a reference image and use the set reference image. The reference image may be referred to as a kind of reference information described above.

The reference image may be referred to as an image in a state in which the object OB does not exist in the capturing area corresponding to the camera module 10 .

In order to set a reference image, the camera module 10 may capture an image of the surroundings as in step S100 of FIG. 8 . In this case, the image captured by the camera module 10 may be referred to as a kind of sample image.

Hereinafter, an image captured by the camera module 10 to generate a reference image may be referred to as a sample image.

In addition, the sample image may be referred to as an image including a vacancy area (or vacancy area). Here, the blank area may be referred to as an area where at least a part of the capturing area is exposed because the object OB is not located in the image captured by the camera module 10 .

When the camera module 10 captures an image, ie, a sample image, in step S100 of FIG. 8 , the captured sample image may be transmitted to the server 20 .

Then, the reference information unit 250 of the server 20 may analyze the sample image transmitted from the camera module 10 to determine whether the sample image includes a blank area (S150).

As a result of determination in step S150, when the blank area is not included in the image transmitted from the camera module 10, a preset second function (Default 2) may be performed.

Here, the second function may be, for example, a function of notifying that the corresponding image is not a sample image including a blank area.

On the other hand, as a result of determination in step S150, if the image transmitted from the camera module 10 includes a blank area, the image may be set as a sample image.

In addition, a base image may be set by combining images of blank areas detected from sample images (S170), and the set base image may be stored (S180).

As shown in (A) of FIG. 17 , in the image captured by the camera module 10, an area in which the object OB does not exist in at least a part of the captured area, that is, a first empty area (First Vacancy Area, VA1) Suppose that includes

In this case, the image shown in (A) of FIG. 17 may be referred to as a first sample image.

The second sample image of FIG. 17 (B) may include a second blank area (VA2) in the left portion of the captured area in the captured image based on the camera module 10 .

The third sample image of FIG. 17 (C) may include a third empty area (VA3) at an upper portion of the captured area in the captured image based on the camera module 10 .

The fourth sample image of FIG. 17 (D) may include a fourth empty area (VA4) at the upper left portion of the captured area in the captured image based on the camera module 10 .

The reference information unit 250 of the server 20 may analyze at least one sample image, preferably a plurality of sample images, and detect a blank area in each sample image.

Thereafter, a reference image may be generated by combining images of blank areas detected from sample images.

An example of a reference image is shown in FIG. 18 .

The reference image of FIG. 18 is the first blank area of the first sample image, the second sample image, the third sample image, and the fourth sample image of (A), (B), (C), and (D) of FIG. 17 ( VA1), the second blank area VA2, the third blank area VA3, and the fourth blank area VA4 may be combined images.

In this way, if the reference image is set corresponding to each camera module 10, sufficiently accurate crowd counting may be possible regardless of the surrounding environment and installation environment of the camera module 10.

For example, depending on the installation position of the camera module 10, the amount of light, the angle of the light, the curved state of the floor of the photographing area, the field of view of the camera module 10, and the like may change.

In consideration of this, in the present invention, after installing the camera module 10 , a reference image in which an object OB does not exist in the photographing area may be independently set corresponding to each camera module 10 . In this case, crowd counting may be performed effectively and precisely in response to the surrounding environment or installation environment of each camera module 10 .

Meanwhile, the reference image may be updated, added, deleted, or updated periodically and/or non-periodically. In detail, it may be possible to periodically and/or non-periodically update, add, delete, or update the reference image by using artificial intelligence in the server 20 .

In this regard, referring to the attached FIG. 19, it will be described below.

Referring to FIG. 19 , after setting a reference image, the camera module 10 may capture an image of the surroundings (S300).

Here, the image captured by the camera module 10 may be referred to as a target image or a sample image.

Thereafter, the image captured by the camera module 10 is transmitted to the server 20, and the server 20 determines whether or not there is a part that is changed differently from the preset reference image in the image received from the camera module 10. It can be judged (S310).

In detail, the server 20 may compare a plurality of images continuously photographed by the camera module 10 according to a predetermined period with a preset and stored reference image to determine whether there is a changed part.

Here, the images used by the server 20 for comparison with the reference image may be images captured in a state in which the photographing angle and photographing area of the camera module 10 are the same as those of the reference image.

As a result of the determination in step S310, when there is no change in the image captured by the camera module 10, the preset reference image may be maintained without being changed (S350).

On the other hand, if there is a changed part in the image captured by the camera module 10 as a result of the determination in step S310, it may be determined whether the data change rate of the changed part is equal to or less than a preset reference change rate (S320).

As a result of determination in step S320, if the change rate of the changed part in the image captured by the camera module 10 is greater than the preset reference change rate, the reference image may be maintained without being changed (S350).

On the other hand, as a result of the determination in step S320, if the change rate of the changed part in the image captured by the camera module 10 is less than the preset reference change rate, it is possible to determine whether the time for which the changed part is maintained is equal to or longer than the preset reference time. Yes (S330).

As a result of the determination in step S330, if the holding time of the changed part in the image captured by the camera module 10 is equal to or longer than the preset reference time, the reference video may be maintained without being changed (S350).

On the other hand, as a result of the determination in step S330, if the holding time of the changed part in the image captured by the camera module 10 is longer than the preset reference time, the existing reference video can be changed to a new reference video and set (S340).

A method of setting a new reference image has been described in detail with reference to FIGS. 17 to 18 above.

As shown in (A) of FIG. 20 , it is assumed that a statue is newly erected in the photographing area of the camera module 10 .

In this case, images continuously photographed by the camera module 10 at regular intervals may commonly include a statue.

In addition, the rate of change of data corresponding to the phase in the image may be less than a preset reference rate of change. In addition, the statue in the image may be included in the image for a predetermined reference time or longer. This can be attributed to the fact that there is almost no movement of the statue.

In this case, the server 20 may recognize the statue as a newly added background object BOB.

Thereafter, the server 20 may create and set a new reference image so that the background object BOB is included in the base image.

20(B) shows an example of a case where a boundary stone on a road is newly installed. Even in this case, the server 20 may be able to recognize the boundary stone of the road as the background object BOB.

In this way, when the server 20 periodically or non-periodically changes the reference image, crowd counting can be performed more effectively and precisely in response to environmental changes in the photographing area corresponding to the camera module 10 .

As described above, the crowd counting mode may be executed after setting reference information, reference images, comparison images, and the like in the server 20 . Referring to the accompanying drawing for the crowd counting mode, it is as follows.

21 to 23 are diagrams for explaining the canonical coefficient mode. Hereinafter, description of the parts described in detail above may be omitted.

Here, the crowd counting mode calculates the number of crowds in an image captured by the camera module 10 in order to detect the number of objects OB located in a predetermined area after the server 20 is set for crowd counting. It can be called detection mode.

From another point of view, in the crowd counting mode, the camera module 10 captures an image (target image) of a predetermined shooting area, and the server 20 captures an object (target image) in the image (target image) captured by the camera module 10. OB) can be detected.

Referring to FIG. 21 , in the crowd counting mode, the camera module 10 may capture an image corresponding to a surrounding area, that is, a capturing area (S400).

In this crowd counting mode, an image captured by the camera module 10 for crowd counting may be referred to as a target image or a target image. Hereinafter, an image captured for crowd counting in the crowd counting mode may be referred to as a target image.

Thereafter, the camera module 10 may transmit an image for crowd counting, that is, a target image to the server 20 .

Then, the server 20 may compare the target image received from the camera module 10 with a preset reference image (S410).

Thereafter, the server 20 may determine whether there is a part different from the reference video, that is, a changed part in the target image (S420).

As a result of the determination in step S420, when a different part of the target image compared to the reference image, that is, there is no changed part, a preset third function (Default 3) may be performed (S430).

Here, the third function is a function of notifying that the number of objects (OB) included in the target image is 0, that is, a function of notifying that the object (OB) is not included in the target image, a function of notifying that the target image is to be captured again, and the like, for example. can be heard

On the other hand, as a result of the determination in step S420, if there is a different part, that is, a changed part, in the target image compared to the reference image, the server 20 may detect the changed part in the target image (S440).

Thereafter, the server 20 may determine an object from the detected changed part (S450).

Thereafter, the server 20 compares the detected changed part with at least one comparison image (or comparison block) previously set to correspond to the object (S460), and compares the object OB in the detected changed part according to the comparison result. The number can be determined (S470).

Thereafter, it is possible to output the final result value by adding up the number of objects OB determined. Here, a predetermined error may be set in the final result value according to circumstances.

For example, assume that the camera module 10 captures an image as shown in FIG. 22 .

The camera module 10 may transmit the image of FIG. 22 to the server 20 in order to detect the number of objects OB included in the image of FIG. 22 . In this case, the image of FIG. 22 may be referred to as a target image.

Then, the server 20 may set the image of FIG. 22 as a target image and compare the target image with a preset reference image.

An example of the reference image is shown in FIG. 18 above.

For example, the server 20 may compare a base image as shown in FIG. 18 and a target image as shown in FIG. 22 .

In addition, as a result of the comparison, the server 20 may detect a part different from the base image in the target image, that is, a changed part.

In FIG. 22, the changed parts are indicated as a fourth area AR4, a fifth area AR5, a sixth area AR6, a seventh area AR7, an eighth area AR8, and a ninth area AR9. .

Such a changed portion may be referred to as a region of interest.

The server 20 may detect each object OB in the detected changed part (region of interest). In the process of detecting the object OB, the server 20 may use preset reference information.

For example, the server 20 may include location information corresponding to the capturing area, location information of the camera module 10, distance information between each location of the capturing area and the camera module 10, shape and key of the object OB. It is possible to detect the object OB in the changed part using various reference information such as information such as etc.

The object OB detection method using reference information has been described in detail above.

Here, an image block may be set to surround the periphery of the objects OB in each region of interest (changed portion) in which the object OB is detected, except for the region of interest (changed portion) in which the object OB is not detected. have.

An image block set in the region of interest of the target image may be referred to as a target block.

For example, as shown in FIG. 22, the first target block TB1 is set in the fourth area AR4, the second target block TB2 is set in the fifth area AR5, and the sixth In the area AR6, it is possible to set the third target block TB3.

Thereafter, the comparison image unit 260 of the server 20 compares the image pattern of the first target block TB1 with at least one pre-stored comparison block, and obtains at least one image pattern having a similarity ratio higher than a preset threshold ratio. A comparison block can be detected. Here, the detected comparison block may be referred to as a candidate comparison block.

Thereafter, one comparison block most similar to the image pattern of the first target block TB1 may be detected from among the at least one candidate comparison block detected. Here, the detected comparison block may be referred to as a matching comparison block.

Thereafter, the number of objects OB included in the matching comparison block may be determined, and the number of objects OB included in the matching comparison block may be determined as the number of objects OB included in the first target block TB1. The number of objects OB included in the first target block TB1 may correspond to the number of objects OB included in the fourth area AR4.

Here, the candidate comparison block and the matching comparison block may be different terms used to help understand the comparison block as a type.

In this way, the number of objects OB included in the fifth area AR5, the sixth area AR5, the seventh area AR7, the eighth area AR8, and the ninth area AR9 is detected, Information on the total number of objects OB included in the target image (final result value) may be detected by summing the number of detected objects OB.

In addition, an error may be given to the total number of objects OB included in the target image (final result value) according to circumstances.

For example, when an abnormal pattern is included in the target image, a predetermined error may be given to the final result value. The abnormal pattern will be described below.

For example, an error may be given, such as 100 people (final result value) ㅁ 5 people (error). Alternatively, it is possible that the error is given as a ratio.

One object OB may be included in the seventh area AR7 , the eighth area AR8 , and the ninth area AR9 .

Target blocks corresponding to the seventh area AR7 , the eighth area AR8 , and the ninth area AR9 may match a comparison block corresponding to one object OB.

Meanwhile, when the moving speed of the objects OB is relatively fast, the shapes of target blocks included in the target image may also be rapidly changed.

For example, suppose that a target image as shown in (A) of FIG. 23 is captured at a first predetermined viewpoint.

Here, at a second viewpoint after the first viewpoint, a target image as shown in (B) of FIG. 23 may be captured.

Comparing (A) and (B) of FIG. 23 , it can be confirmed that the image pattern of the target block is changed as the objects OB move.

This phenomenon may be further intensified when the moving speed of the objects OB is high.

Considering this, it is possible to detect the moving speed of the objects OB according to the degree to which the image pattern of the target block of the target image is changed. Here, the moving speed of the objects OB can be used to determine the floating population in a certain area.

As described in detail above, a similar comparison image is detected by comparing a target image for which the number of objects OB is to be determined with a previously set and stored comparison image, and the number of objects OB included in the detected comparison image is calculated. It can be determined as the number of objects (OB) included in the target image.

Alternatively, a target block included in a target image to determine the number of objects OB is compared with a previously set and stored comparison block to detect a similar comparison block, and the number of objects OB included in the detected comparison block is calculated. It can be determined as the number of objects (OBs) included in the target block. Thereafter, the number of objects OB included in the target image may be determined as the number of objects OB included in the target image by summing the number of objects OB of all target blocks included in the target image.

In the present invention, the number of objects OB included in the target image can be determined more quickly.

24 to 29 are views for explaining a camera module. Hereinafter, description of the parts described in detail above may be omitted.

Referring to FIG. 24 , the camera module 10 may include a first camera part 11 and a second camera part 12 .

Here, at least a portion of the capturing areas of the first camera unit 11 and the second camera unit 12 may be different from each other.

For example, the photographing area corresponding to the first camera unit 11 includes a first non-overlap area (NOVA1) and an overlap area (OVA), and the second camera unit 12 The photographing area corresponding to may include a second non-overlap area (NOVA2) and an overlapping area (OVA).

Here, the overlapping area OVA may be an area where a capturing area corresponding to the first camera unit 11 and a capturing area corresponding to the second camera unit 12 overlap each other.

The first non-overlapping area NOVA1 may be a remaining area except for the overlapping area OVA in the capturing area corresponding to the first camera unit 11 .

The second non-overlapping area NOVA2 may be an area remaining after excluding the overlapping area OVA from the capturing area corresponding to the second camera unit 12 .

Here, the size of the overlapping area OVA may be greater than the sum of the sizes of the first non-overlapping area NOVA1 and the size of the second non-overlapping area NOVA2.

In detail, the area of the overlapping area OVA may be sufficiently greater than the sum of the areas of the first non-overlapping area NOVA1 and the area of the second non-overlapping area NOVA2 .

When the capturing area corresponding to the first camera unit 11 and the capturing area corresponding to the second camera unit 12 overlap sufficiently, the first non-overlapping area NOVA1 and the second non-overlapping area NOVA2 are disregarded. It may be possible to do

As such, when the camera module 10 includes a plurality of camera units, it may be possible to more precisely and quickly detect the number of objects OB.

Here, at least one of the first camera unit 11 and the second camera unit 12 may rotate, pivot or move.

Alternatively, each of the first camera unit 11 and the second camera unit 12 may be fixed to a predetermined frame.

When the camera module 10 includes the first camera unit 11 and the second camera unit 12, the server 20 stores the target image captured by the first camera unit 11 and the second camera unit 12. It is possible to detect the number of objects OB by analyzing each of the target images taken by ). In this regard, referring to the accompanying drawings, it will be described as follows.

Referring to FIG. 25 , the number of objects OB corresponding to the first camera unit 11 is detected (S471), and the number of objects OB corresponding to the second camera unit 12 is detected (S472). can

In other words, the number of objects OB is detected in the image (target image) captured by the first camera unit 11, and the number of objects OB is detected in the image (target image) captured by the second camera unit 12. number can be detected.

Hereinafter, a target image captured by the first camera unit 11 may be referred to as a first target image, and an image captured by the second camera unit 12 may be referred to as a second target image.

For example, the second controller 200 of the server 20 may transmit a command to the camera module 10 to detect the number of objects OB located in the capturing area.

In this case, it can be said that the crowd counting mode is activated. The crowd counting mode has been described in detail above.

In the crowd counting mode, the first camera unit 11 and the second camera unit 12 of the camera module 10 may each capture an image (target image) for the capturing area.

When the first camera unit 11 and the second camera unit 12 respectively capture images (target images), it is possible to detect the number of objects OB in each image.

The method of detecting the number of objects OB in the target image has been described in detail in the foregoing FIG. 21 and the corresponding description.

It is possible for the first camera unit 11 and the second camera unit 12 to capture the target image at the same time point.

For example, the server 20 may transmit a command to capture a target image for each corresponding capturing area at the 10th viewpoint to the camera module 10 .

Then, in the camera module 10, the first camera unit 11 and the second camera unit 12 may each capture a target image at the tenth viewpoint.

Alternatively, a capturing time point of the first camera unit 11 and a capturing time point of the second camera unit 12 may be different from each other.

For example, the server 20 may transmit a command to capture a target image for each corresponding capturing area at the 10th viewpoint to the camera module 10 .

Then, in the camera module 10, the first camera unit 11 captures the first target image of the corresponding capturing area at the 10th viewpoint, and the second camera unit 12 captures the corresponding photographing at the 10+Δt viewpoint. A second target image of the region may be captured.

It may be difficult enough for the human object OB to maintain a perfectly still state like an inanimate object, and may have a certain degree of movement. For example, during a time period of Δt, at least one object OB may perform a movement such as moving or changing a pose.

Considering this, the position, pose, shape, size, etc. of at least one object OB in the first target image captured by the first camera unit 11 and the second target image captured by the second camera unit 12 this may vary.

Alternatively, the overlapping pattern of the plurality of objects OB may be changed in the first target image captured by the first camera unit 11 and the second target image captured by the second camera unit 12 .

Considering this, when the capturing time of the first camera unit 11 and the capturing time of the second camera unit 12 are different in the crowd counting mode, the number of objects OB located in the capturing area is more accurately detected. can do.

25, after detecting the number of objects OB for the first target image and detecting the number of objects OB for the second target image, it is possible to determine whether the detected result values are the same ( S473).

Here, the result value may mean a result value for the number of objects OB detected for the first target image and a result value for the number of objects OB detected for the second target image.

As a result of the determination in step S473, if the result values are the same, the result value corresponding to either one of the first camera unit 11 and the second camera unit 12 is output as the final result value for the number of objects OB. It can (S474).

For example, as a result of analyzing the first target image captured by the first camera unit 11 in the server 20, it may be determined that the number of objects OB included in the first target image is 20 people. In addition, as a result of analyzing the second target image captured by the second camera unit 12 in the server 20, it may be determined that the number of objects OB included in the second target image is 20 people.

In this case, the server 20 can finally output the number of objects OB located in the photographing area corresponding to the camera module 10 to 20. That is, the server 20 may be able to output a result value corresponding to any one of the first camera unit 11 and the second camera unit 12 as a final result value.

On the other hand, as a result of the determination in step S473, if the result values are not the same and different, the reliability of the first camera unit 11 and the second camera unit 12 may be evaluated (S475).

Then, it may be determined whether the reliability of the first camera unit 11 and the reliability of the second camera unit 12 each exceed preset threshold reliability (S476).

As a result of determination in step S476, if the reliability of the first camera unit 11 and the reliability of the second camera unit 12 are each equal to or greater than the preset threshold reliability, the server 20 captures the first image taken by the first camera unit 11. The number of objects OB may be detected (object OB counting) in each of the first target image and the second target image captured by the second camera unit 12 (S477).

Thereafter, the server 20 sums the number of objects OB included in the first target image and the number of objects OB included in the second target image, and divides the summed value by the number of cameras (2). An average value (median value) can be obtained (S478).

Then, the server 20 may be able to output the average value (median value) as the final result value.

For example, as a result of analyzing the first target image captured by the first camera unit 11 in the server 20, it is determined that the number of objects (OB) included in the first target image is 20, and the server As a result of analyzing the second target image captured by the second camera unit 12 in (20), it can be determined that the number of objects OB included in the second target image is 30 people.

Here, the reliability of the first camera unit 11 and the reliability of the second camera unit 12 may each be greater than or equal to the threshold reliability.

In this case, the detection unit 280 of the server 20 may determine that the number of objects OB located in the photographing area corresponding to the camera module 10 is 25 people.

Meanwhile, as a result of determination in step S476, if the reliability of at least one of the first camera unit 11 and the second camera unit 12 is lower than the preset threshold reliability, the server 20 determines the first camera unit ( 11) and the second camera unit 12, a target image corresponding to one of higher reliability may be selected (S479).

Thereafter, the server 20 may output information on the number of objects OB corresponding to the selected target image as a final result (S480).

For example, as a result of analyzing the first target image captured by the first camera unit 11 in the server 20, it is determined that the number of objects (OB) included in the first target image is 20, and the server As a result of analyzing the second target image captured by the second camera unit 12 in (20), it can be determined that the number of objects OB included in the second target image is 30 people.

Here, it may be possible that the reliability of the first camera unit 11 is higher than the preset threshold reliability and the reliability of the second camera unit 12 is lower than the threshold reliability.

Alternatively, the reliability of the first camera unit 11 and the reliability of the second camera unit 12 are each lower than the threshold reliability, wherein the reliability of the first camera unit 11 is higher than the reliability of the second camera unit 12. could be higher

In this case, the server 20 selects the first target image corresponding to the first camera unit 11 having a relatively high reliability, and information 20 on the number of objects OB included in the selected first target region. name) as the final result.

The reliability of the first camera unit 11 and the second camera unit 12 will be examined.

As shown in (A) of FIG. 26, it is assumed that four objects (OB1, OB2, OB3, and OB4) are located side by side in a row.

Here, the first object OB1 is in a standing state, the second object OB2 located behind the first object OB1 is shorter than the first object OB1, and the second object OB2 is taller than the first object OB1. The third object OB3 located behind may be a type sitting on a chair, and the fourth object OB4 located behind the third object OB3 may be a type sitting on the floor.

In this case, as shown in (B) of FIG. 26, the distances between the first object OB1, the second object OB2, the third object OB3, and the fourth object OB4 are sufficiently small, respectively. , the first object OB1 , the second object OB2 , the third object OB3 , and the fourth object OB4 may be located side by side in front of the first camera unit 11 .

Then, from the viewpoint of the first camera unit 11, the first object OB1, the second object OB2, the third object OB3, and the fourth object OB4 overlap each other, and accordingly, the object OB It can be difficult to pinpoint the exact number.

On the other hand, from the viewpoint of the second camera unit 12, the sides of the first object OB1, the second object OB2, the third object OB3, and the fourth object OB4 can be observed, so that the first object (OB1), the second object (OB2), the third object (OB3), and the fourth object (OB4) can be classified relatively precisely.

Considering this, in the case of FIG. 26 , the reliability of the second camera unit 12 may be higher than that of the first camera unit 11 .

From another point of view, in the case of FIG. 26 , the reliability of the second camera unit 12 may be greater than or equal to a preset threshold reliability, and the reliability of the first camera unit 11 may be lower than the threshold reliability.

In the case of FIG. 26 , the first object OB1, the second object OB2, the third object OB3, and the fourth object OB4 overlap with the target image captured by the first camera unit 11. Therefore, if the image block is set to surround the outer periphery of the objects OB, the pattern of the set image block may be similar to the pattern of the image block including one object OB.

However, when the second object OB2, the third object OB3, and the fourth object OB4 located behind the first object OB1 do not completely overlap with the first object OB1, , At least a part of at least one of the second object OB2, the third object OB3, and the fourth object OB4 may be exposed beyond the first object OB1. For example, at least one arm or leg among the second object OB2, the third object OB3, and the fourth object OB4 may appear around the first object OB1.

As such, an image pattern when at least one object OB is sufficiently covered by another object OB may be referred to as an abnormal pattern. In detail, a case where at least one object OB is obscured by another object OB at a predetermined reference rate or more may be referred to as an abnormal pattern. Here, the reference ratio may be determined between approximately 95 and 99%.

For example, in the case of FIG. 26 , when approximately 97% of the second object OB2 is covered by the first object OB1 from the viewpoint of the first camera unit 11, the first camera unit ( It can be seen that the target image captured by 11) includes an abnormal pattern.

In such an abnormal pattern, a possibility that it is difficult to accurately detect the number of objects OB may be relatively high.

Therefore, it may be possible to determine reliability based on whether or not an abnormal pattern is included.

For example, as in the case of (B) of FIG. 26, an image (first target image) captured by the first camera unit 11 includes an abnormal pattern, and an image captured by the second camera unit 12. When the abnormal pattern is not included in the (second target image), it may be determined that the reliability of the second camera unit 12 is higher than that of the first camera unit 11 . Alternatively, the reliability of the second camera unit 12 may be greater than or equal to a preset threshold reliability, and the reliability of the first camera unit 11 may be lower than the threshold reliability due to an abnormal pattern.

Alternatively, reliability may be determined according to the number of abnormal patterns included in the target image.

For example, if the image captured by the first camera unit 11 (the first target image) includes more abnormal patterns than a preset reference number (eg, 5), the second camera unit 12 When the image (second target image) captured by the camera includes fewer than the reference number of abnormal patterns, it is determined that the reliability of the second camera unit 12 is higher than that of the first camera unit 11, or , it may be determined that the reliability of the second camera unit 12 is greater than or equal to a preset threshold reliability and the reliability of the first camera unit 11 is lower than the threshold reliability due to the abnormal pattern.

Even when at least one abnormal pattern is included in the target image, which is a target for detecting the number of objects OB, the corresponding abnormal pattern is compared with a preset comparison block, preferably overlapping blocks, and objects included in the abnormal pattern (OB) are compared. ) can be quickly detected.

The overlapping block has been previously described in detail with reference to FIGS. 15 and 16 .

Another example of an anomalous pattern is shown in FIG. 27 .

As shown in (A) of FIG. 27, assume a case in which four objects OB1, OB2, OB3, and OB4 are in close enough contact with each other in a line.

Here, the first object OB1 , the second object OB2 , the third object OB3 , and the fourth object OB4 may be in close contact with each other to the extent that their bodies come into contact with each other.

In this case, as shown in (B) of FIG. 27, the first object OB1, the second object OB2, the third object OB3, and the fourth object OB4 are the first camera unit 11 ) can be located side by side in front of.

Then, from the viewpoint of the first camera unit 11 , the first object OB1 , the second object OB2 , the third object OB3 , and the fourth object OB4 may overlap each other.

On the other hand, from the viewpoint of the second camera unit 12, the sides of the first object OB1, the second object OB2, the third object OB3, and the fourth object OB4 can be observed, so that the first object (OB1), the second object (OB2), the third object (OB3), and the fourth object (OB4) can be classified relatively accurately.

Considering this, in the case of FIG. 27 , the reliability of the second camera unit 12 may be higher than that of the first camera unit 11 .

In addition, an abnormal pattern may be included in an image (target image) captured by the first camera unit 11 .

Meanwhile, while the reliability of the first camera unit 11 and the second camera unit 12 is higher than the preset threshold reliability, the number of objects OB corresponding to the first target image and the number of objects corresponding to the second target image Let's assume a case where the number of objects (OBs) to do is different.

As shown in FIG. 28 , it is assumed that the first camera unit 11 and the second camera unit 12 each capture a portion where a plurality of objects OB are gathered.

An image pattern of a portion where a plurality of objects OB are gathered may be determined differently according to a photographing angle.

For example, in the case of FIG. 28 , the server 20 has an image pattern of a portion (target block) where a plurality of objects OB are gathered in an image (first target image) captured by the first camera unit 11. It may be determined that the image pattern of the preset first comparison block is the most similar.

On the other hand, the server 20 uses a second comparison block in which an image pattern of a part (target block) where a plurality of objects OB are gathered in an image (second target image) captured by the second camera unit 11 is set in advance. It can be judged to be most similar to the image pattern of .

In this case, the server 20 analyzes the first target image corresponding to the first camera unit 11, outputs a first result value for the number of objects OB included in the first target image, and A second result value may be output by analyzing the second target image corresponding to the camera unit 12 to detect the number of objects OB included in the second target image.

Thereafter, the server 20 may sum up the first result value and the second result value, and output a final result value by averaging.

In this case, it is possible to suppress or prevent the occurrence of errors according to the photographing angle.

Meanwhile, the capturing area corresponding to the first camera unit 11 and the capturing area corresponding to the second camera unit 12 partially overlap, and the non-overlapping area corresponding to the first camera unit 11 and the second camera unit 12 overlap. If the size of the non-overlapping area corresponding to (12) is sufficiently large, counting the number of objects OB in the overlapping area and the non-overlapping area may be performed in a different way. In this respect, referring to the accompanying drawings, the following.

The overlapping area and the non-overlapping area have been described in detail with reference to FIG. 24 above.

Referring to FIG. 29 , the number of objects OB in the overlapping area OVA may be detected (S490).

The overlapping area OVA is an area where the capturing area corresponding to the first camera unit 11 and the capturing area corresponding to the second camera unit 12 overlap, and the first camera unit 11 and the second camera unit ( 12) can be said to be an area where shooting is performed.

The first camera unit 11 transmits the first target image capturing the corresponding capturing area to the server 20, and the second camera unit 12 transmits the second target image capturing the corresponding capturing region to the server 20 ), the server 20 may determine an overlapping region OVA in each of the first target image and the second target image.

In addition, the server 20 detects the number of objects OB in the overlapping area OVA of the first target image and detects the number of objects OB in the overlapping area OVA of the second target image. can

Thereafter, the server 20 selects information on the number of objects OB with respect to the overlapping area OVA corresponding to any one of the first camera unit 11 and the second camera unit 12 based on the reliability. Alternatively, it is possible to calculate the average value.

For example, suppose that the reliability of the first camera unit 11 is higher than that of the second camera unit 12 or the reliability of the first camera unit 11 is higher than a preset threshold reliability. .

In this case, the server 20 determines an overlapping area (OVA) in the first target image corresponding to the first camera unit 11, and determines the overlapping area (OVA) of the object (OB) in the determined overlapping area (OVA) of the first target image. The number may be detected, and information on the number of detected objects OB may be determined as a final result value corresponding to the overlapping area OVA.

Alternatively, the server 20 determines the overlapping area OVA in the first target image corresponding to the first camera unit 11, and the number of objects OB in the determined overlapping area OVA of the first target image. (first result) can be detected. In addition, the server 20 determines the overlapping area OVA in the second target image corresponding to the second camera unit 12, and the number of objects OB in the determined overlapping area OVA of the second target image. (Second result) can be detected.

Thereafter, the server 20 may sum up and average the first result and the second result, and output a final result value corresponding to the overlapping area OVA.

Since the contents of this can be fully explained from the above detailed description, redundant descriptions can be omitted.

Thereafter, a predetermined first error may be applied to the final result value of the overlapping area OVA (S491).

Here, it may be possible to omit the first error according to circumstances.

Meanwhile, the number of objects OB can be detected even in a non-overlapping area (S492).

For example, the server 20 detects the number of objects OB in the first non-overlapping area NOVA1 corresponding to the first camera unit 11, and also corresponds to the second camera unit 12. The number of objects OB may be detected for the second non-overlapping area NOVA2.

In addition, the server 20 calculates the sum of the number of objects OB corresponding to the first non-overlapping area NOVA1 and the number of objects OB corresponding to the second non-overlapping area NOVA2. It can be determined by the final result of the number of objects OB corresponding to the areas NOVA1 and NOVA2.

Thereafter, a predetermined second error may be given to final result values for the non-overlapping areas NOVA1 and NOVA2 (S493).

Here, it may be possible to omit the second error according to circumstances.

In a situation where both the first error and the second error are given, the second error may be greater than the first error in a ratio of the error value to the final result value.

For example, assuming that the final result value for the overlapping area (OVA) is 100 people, the first error may be ㅁ3% as ㅁ3% for 100 people.

On the other hand, assuming that the final result values for the non-overlapping areas NOVA1 and NOVA2 are 100 people, the second error may be ㅁ10% of 100 people, which is ㅁ10 people.

In this way, the reason why the ratio of the error (second error) to the final result value of the non-overlapping areas NOVA1 and NOVA2 is larger than the ratio of the error (first error) to the final result value of the overlapping area OVA is This may be because the probability of occurrence of an error is relatively high because the non-overlapping areas NOVA1 and NOVA2 are photographed by the first camera unit 11 or the second camera unit 12 alone.

After discriminating final result values and errors for the non-overlapping areas NOVA1 and NOVA2 and the overlapping areas OVA, respectively, the final result values for the camera module 10 may be output by summing the two values ( S494).

For example, the final result value for the overlapping area (OVA) is 100 people, the first error is ㅁ3 people, the final result value for the non-overlapping area (NOVA1, NOVA2) is 100 people, and the second error is Let's assume the case of 10 people.

In this case, the overall final result value may be 200 people, and the error may be ㅁ13 people.

In the present invention, since the number of objects OB included in the target image is determined by comparing the target image (target block) and the comparison image (comparison block), the number of objects OB can be determined and detected more quickly. can That is, crowd counting can be performed more quickly.

Accordingly, the present invention may be more preferably applied to a case where it is necessary to rapidly determine the number of objects OB.

In addition, in the present invention, it is possible to quickly detect the number of objects OB even when using low-end hardware and software.

In addition, in the present invention, it may be more effective to quickly determine the number of objects OB included in a relatively narrow space.

For example, the present invention can be more usefully used to detect the number of objects OB (passengers) located in a train such as a subway.

Trains such as subways have a relatively narrow space, and a large number of objects (OBs) may be located in a narrow space according to the number of passengers, and the number of objects (OBs) located in the train in response to getting on and off is frequent. may change accordingly.

A method of installing a camera module 10 in a train and detecting the number of objects (OB) located in the train, that is, the number of passengers, using the camera module 10 is as follows with reference to the accompanying drawings.

30 to 32 are diagrams for explaining a case in which a camera module is installed in an electric train. Hereinafter, description of the parts described in detail above may be omitted.

Referring to FIG. 30 , the camera module 10 may be installed inside an electric train 30 such as a subway. For example, the camera module 10 may be installed on an inner upper wall of the train 30 .

In this case, it is possible to install the camera module 10 on the upper wall of the central part of the train 30 in order to suppress or prevent the occurrence of errors according to the distance.

As such, when the camera module 10 is installed in the center of the train 30, the first camera unit 11 and the second camera unit 12 of the camera module 10 may face different directions.

For example, as shown in FIG. 31, the first camera unit 11 of the camera module 10 is inclined toward the left side of the train 30, and the second camera unit 12 is the train ( 30) may be tilted obliquely toward the right part.

In addition, considering the characteristics of the train 30 having a narrow and long space, the size (area) of the first non-overlapping area NOVA1 of the shooting area corresponding to the first camera unit 11 is the overlapping area OVA ), and the size (area) of the second non-overlapping area NOVA2 of the capturing area corresponding to the second camera unit 12 is greater than the size (area) of the overlapping area OVA. can

In this way, the camera module 10 is installed in the train 30, and the server 20 uses the image (target image) captured by the camera module 10 to create an object (OB) located in the train 30, that is, The number of passengers can be detected.

In addition, in the train 30, objects OB, that is, passengers, are located in a space having a long and narrow shape, and movement of the objects OB may be relatively reduced while the train 30 is moving.

Considering this, as in the present invention, a technology for quickly detecting the number of objects (OB) included in a target image (target block) by comparing a target image (target block) and a comparison image (comparison block) is a train 30 ) may be more preferable.

In the case where the camera module 10 is applied to the train 30, a method of performing monarch counting will be described with reference to the accompanying drawings.

In order to perform crowd counting in the train 30, the server 20 and the train control unit (not shown) controlling the operation of the train 30 may interwork with each other and perform communication.

Referring to FIG. 32, first, the opening and closing of the door of the train 30 can be confirmed (S500).

Subsequently, it may be determined whether or not the door of the train 30 is closed (S510).

As a result of the determination in step S510, when the door of the train 30 is not closed, that is, when the door is open, a preset fourth function (Default 4) can be performed (S520).

The fourth function may include, for example, a function of waiting until the door of the train 30 is closed.

On the other hand, as a result of the determination in step S510, in a state where the door of the train 30 is closed, the speed of the train 30 can be checked (S530).

Thereafter, it may be determined whether or not the speed of the train 30 is equal to or greater than a preset threshold speed (S540).

As a result of determination in step S540, if the speed of the train 30 is lower than the critical speed, a preset fifth function (Default 5) can be performed (S550).

The fifth function may be, for example, a function of waiting until the speed of the train 30 exceeds a critical speed.

On the other hand, if the speed of the train 30 exceeds the critical speed as a result of the determination in step S550, the first camera unit 11 and the second camera unit 12 of the camera module 10 are used to capture an image of the area, That is, the target image may be photographed (S400).

Here, the shooting area may include the inside of the train 30, and the target image may include an image of the inside of the train 30.

Thereafter, the object OB, that is, the number of passengers may be detected from the target image captured by the camera module 10 (S470).

Since the camera module 10 captures the target image of the capturing area and the method of detecting the number of objects OB in the target image has been described in detail above, duplicate descriptions will be omitted.

After detecting the number of objects OB in the target image, it is determined whether to end the crowd counting mode (S560), and if so, the crowd counting mode can be ended (S570).

On the other hand, as a result of determination in step S560, if the crowd counting mode is not terminated, the process may proceed to step S500 again.

Considering the contents of FIG. 32 , the camera module 10 may not take pictures when the speed of the train 30 is below the critical speed or when the door of the train 30 is open.

On the other hand, when the speed of the train 30 is higher than the critical speed while the door of the train 30 is closed, the camera module 10 captures an image and detects the number of objects OB in the captured image. can In addition, this process may be repeated.

From another point of view, in the case of a subway, the camera module 10 may not take pictures while the train 30 stops at a station and passengers get on or off. On the other hand, when the train 30 departs from the station and reaches a sufficient speed (when the threshold speed is exceeded), the camera module 10 captures an image and detects the number of objects OB in the captured image. can

This process may be performed at least once between two adjacent stations. Since there are cases where objects OB, that is, passengers, pass between trains 30, it may be preferable to perform a process of detecting the number of objects OB between two adjacent stops a plurality of times. have.

In addition, the process of detecting the number of objects OB may be performed again after passing a predetermined stop.

As described above, after detecting the number of objects (OB) located inside the train 30, that is, the number of passengers, information on the detected number of passengers is sent to a central server (not shown) that manages the train 30. can transmit

Then, the central server may notify customers waiting to board at the first stop of the number of passengers boarding the train 30 in advance before the train 30 arrives at the first stop.

Then, customers waiting at the first stop can check in advance which train has a small number of passengers, and accordingly, the effect of evenly distributing passengers to the plurality of trains 30 can be obtained.

As such, it will be understood that the technical configuration of the present invention described above can be implemented in other specific forms without changing the technical spirit or essential features of the present invention by those skilled in the art to which the present invention pertains.

Therefore, the embodiments described above should be understood as illustrative and not restrictive in all respects, and the scope of the present invention is indicated by the claims to be described later rather than the foregoing detailed description, and the meaning and scope of the claims And all changes or modified forms derived from the equivalent concept should be construed as being included in the scope of the present invention.

Claims (5)

A photographing step of capturing a surrounding image by a camera module; and
a detection step of analyzing a target image captured by the camera module and detecting the number of objects included in the target image;
Crowd counting method using artificial intelligence comprising a. According to claim 1,
The camera module
Including a first camera part and a second camera part,
The crowd counting method using artificial intelligence, characterized in that at least some of the shooting areas of the first camera unit and the second camera unit are different from each other. According to claim 1,
determining and storing positional information about a photographing area of the camera module;
Determining and storing information on the installation location of the camera module; and
determining at least one object in the target image;
Including more,
In the detection step
The crowd counting method using artificial intelligence, characterized in that for detecting the number of the object based on the position information of the object in the target image and the reference information of the object in the position of the object. According to claim 1,
Taking and storing, by the camera module, a comparison image of the surroundings; and
detecting the number of objects in the comparison image;
Including more,
In the detection step
Crowd counting method using artificial intelligence, characterized in that for detecting the number of objects by comparing the target image with at least one comparison image. A camera module for capturing surrounding images; and
a detection unit that analyzes a target image captured by the camera module and detects the number of objects included in the target image;
Crowd counting device using artificial intelligence comprising a.

Images