KR102607174B1 - Counting method of objects included in multiple images using an image analysis server and object counting system - Google Patents

Abstract

The present invention relates to an image analysis server, an object counting method using an image analysis server, and an object counting system.
According to one aspect of the present invention, receiving an image containing one or more objects is input by a user terminal; forming a plurality of boxes for each object by an image analysis server, leaving only the number of boxes corresponding to the object among the plurality of boxes and deleting the remaining boxes; And an object counting method using an image analysis server may be provided, including counting the number of boxes remaining by the image analysis server and transmitting the number corresponding to the box to the user terminal.

Description

Counting method of objects included in multiple images using an image analysis server and object counting system}

The present invention relates to a method and object counting system for counting objects included in a plurality of images using an image analysis server.

As we enter an aging society, the demand of patients visiting hospitals is increasing, and the type and number of drugs administered are also increasing accordingly.

However, when administering pills to patients or conducting inventory checks at small pharmacies or hospitals, there is the inconvenience of having to count the number of pills manually. In addition, when people manually count pills, many errors occur, such as administering less or more than the set number of pills.

To solve this problem, large-scale pharmacies and hospitals are introducing and using pill-counting devices, but these devices are expensive, making it difficult for small-scale pharmacies or hospitals to purchase them.

Additionally, this equipment has the problem of not being able to count multiple types of pills at once.

Patent document: Korea Publication No. 10-2016-0057187 (published on May 23, 2016)

Embodiments of the present invention have been proposed to solve the above problem, and are based on multiple images using an image analysis server that can count the number of objects (e.g., pills) placed on multiple object plates at once. The goal is to provide a counting method and object counting system for included objects.

In addition, we aim to provide a method and object counting system for counting objects included in multiple images using an image analysis server that can simply count the number of objects (e.g., pills) without introducing complex and expensive equipment. .

In addition, we aim to provide a method and object counting system for counting objects included in a plurality of images using an image analysis server that can accurately count the number of closely attached objects (for example, pills).

According to one embodiment of the present invention, selecting a single analysis mode in which one image can be input or a multi-analysis mode in which a plurality of images can be input by the user terminal; When a multi-analysis mode is selected, inputting a plurality of images containing one or more objects by the user terminal and transmitting the plurality of input images to an image analysis server; Counting the number of objects included in each of the plurality of images by the image analysis server; A method of counting objects included in a plurality of images using an image analysis server may be provided, including displaying the number of objects included in each of the plurality of images by the user terminal.

In addition, the step of displaying the number of objects included in each of the plurality of images by the user terminal includes the step of displaying a plurality of images in the multi-analysis window of the user terminal: each of the plurality of images in the multi-analysis window of the user terminal Displaying the number of objects included; Displaying the type of object included in each of the plurality of images on the type display of the user terminal; A method of counting objects included in a plurality of images using an image analysis server may be provided, including displaying the sum of the objects included in all of the plurality of images on the total number display of the user terminal.

In addition, the step of inputting a plurality of images containing one or more objects by the user terminal includes the step of seating the user terminal on the terminal seating portion of the multi-analysis assistance device; A step of seating a plurality of object plates on which objects are placed on a moving belt; Step of sequentially positioning a plurality of object plates at the bottom of the user terminal as the moving belt moves; And each object plate stays at the bottom of the user terminal for a certain period of time and then moves, thereby moving the plurality of object plates to the bottom of the user terminal, and generating a plurality of images by the user terminal photographing the object placed on each object plate. A method of counting objects included in a plurality of images using an image analysis server including a may be provided.

In addition, after the step of seating the plurality of object plates on which the objects are placed on the moving belt, the object plates move so that the objects placed on the object plates pass the front of the sensor of the multi-analysis assistance device. A method for counting objects included in a plurality of images may be provided.

In addition, after the step of counting the number of objects contained in each of the plurality of images by the image analysis server, a step of determining the type of the object by matching the object and the type identification tag by the type determination module of the image analysis server is further performed. A method of counting objects included in a plurality of images using an image analysis server may be provided.

According to another embodiment of the present invention, an object plate that provides a space for placing an object and includes a type identification tag provided in one or more of letters, barcodes, and certain symbols; A user terminal capable of photographing the object plate and receiving an image including one or more objects placed on the object plate and an image including a type identification tag; And an object counting system including an image analysis server capable of determining the number and type of objects included in the image may be provided.

In addition, the object counting system may be provided with a multi-analysis auxiliary device including a terminal seating portion on which a user terminal can be placed at a preset distance from the object plate.

In addition, the object counting system may be provided on which a plurality of object plates are seated and further includes a moving belt capable of moving the plurality of object plates to the bottom of the user terminal.

In addition, the multi-analysis assistance device includes a lower part where the object plate moves; An upper surface portion including a terminal seating portion on which a user terminal can be placed; And an object counting system including a side portion connecting the lower portion and the upper portion may be provided.

In addition, a sensor capable of scanning the heights of a plurality of objects placed on the object plate is provided on one side of the multi-analysis assistant, and the sensor detects objects exceeding a certain range among the objects placed on the object plate. An object counting system capable of sending a warning signal to the user terminal may be provided.

Additionally, an object counting system may be provided in which the side portion is provided with an adjustable length so that the distance between the lower portion and the upper portion can be adjusted.

Additionally, the moving belt may be provided with an object counting system provided to form a closed curve.

The method and object counting system for counting objects included in a plurality of images using an image analysis server according to embodiments of the present invention can count the number of objects (for example, pills) placed on multiple object plates at once. There is an advantage.

In addition, there is an effect of simply counting the number of objects (eg, pills) without introducing complex and expensive equipment.

1 is a diagram schematically showing an object counting system according to an embodiment of the present invention.
FIG. 2 is a diagram schematically showing the configuration of the image analysis server of FIG. 1.
FIG. 3 is a diagram schematically showing the sub-configuration of the second box removal module of the image analysis server of FIG. 2.
Figure 4 is a flowchart schematically showing an object counting method using an image analysis server executed by the object counting system of Figure 1;
FIG. 5 is a flowchart showing step S2 among steps S1 to S3 in FIG. 4 in more detail.
FIG. 6 is a flowchart showing step S36 among steps S32 to S36 of FIG. 5 in more detail.
FIG. 7 is a diagram conceptually showing an object being input by the user terminal shown in FIG. 1.
Figure 8 is a diagram conceptually showing that RetinaNet, an object recognition deep learning model, is executed by the image analysis server of Figure 1 to form a plurality of boxes for one pill.
FIG. 9 is a diagram conceptually showing a box formed in one pill (object) by executing NMS (non maximum suppression), an algorithm for removing boxes, by the image analysis server of FIG. 1.
FIG. 10 is a conceptual diagram to help understand step S36 shown in FIG. 6.
FIG. 11 is a diagram schematically showing a flowchart of transmitting a plurality of images to an image analysis server and counting one or more objects included in each of the plurality of images using the object counting system of FIG. 1;
FIG. 12 is a diagram illustrating a screen displayed on the user terminal of FIG. 1 in single analysis mode and multi-analysis mode.
FIG. 13 is a diagram showing the number and type of objects included in each of a plurality of images for which analysis has been completed by the image analysis server on the screen of the user terminal of FIG. 1 in the multi-analysis mode.
FIG. 14 is a diagram schematically showing a multi-analysis assistance device on which the user terminal of FIG. 1 can be mounted.
FIG. 15 is a diagram schematically showing the multi-analysis auxiliary device and moving belt of FIG. 14 for easily performing the multi-analysis mode.

Figure 1 is a diagram schematically showing an object counting system 1 according to an embodiment of the present invention.

Referring to FIG. 1, the object counting system 1 may include an image analysis server 10, a user terminal 20, and an administrator terminal 30.

Here, the image analysis server 10, the user terminal 20, and the administrator terminal 30 are provided as independent devices and communicate data through the communication network 40, or the image analysis server 10 and the administrator terminal 30 ) is composed of one physical device and can be provided to enable direct data communication.

In this embodiment, the image analysis server 10, the user terminal 20, and the administrator terminal 30 are provided as separate and independent devices.

The object counting system 1 of this embodiment can be understood as a system that can accurately count the number of objects included in an image.

Specifically, when a user photographs an object through the user terminal 20, the image including the photographed object is transmitted to the image analysis server 10, and is converted into the image through an algorithm preset by the image analysis server 10. The number of objects that appear can be counted.

In this embodiment, the object is explained as an example of a pill having a certain shape. When the object captured by the user terminal 20 is a pill, the object counting system 1 of this embodiment can be understood as a system that counts the number of pills available in pharmacies and hospitals.

However, the spirit of the present invention is not limited to this, and the object may include any object having a certain shape.

The image analysis server 10 may be understood as a server that receives image data from the user terminal 20 and processes data necessary to count the number of objects appearing in the image.

Objects included in one image may be the same type of object with the same size and shape. That is, the image analysis server 10 can count the same object included in one image.

However, the spirit of the present invention is not limited to this, and objects included in one image may be different types of objects with different sizes and shapes. In this case, the image analysis server 10 You can also count the different types of objects included.

The user terminal 20 can photograph an object placed on an object plate, which will be described later, and display it as an image.

Additionally, the user terminal 20 is a device capable of communicating with the image analysis server 10 and may be a mobile terminal or a fixed terminal implemented as a computing device.

For example, the user terminal 20 may include a smartphone, a laptop, a tablet PC, a wearable device, a computer, etc. including a camera capable of photographing an object. However, the user terminal 20 is not limited to this example and may be provided as a separate camera.

The administrator terminal 30 can be understood as a device that can update functions provided to the user terminal 20 through the image analysis server 10 or input a certain command. For example, the manager terminal 30 may include a smartphone, laptop, tablet PC, wearable device, computer, etc. capable of communicating with the image analysis server 10.

FIG. 2 is a diagram schematically showing the configuration of the image analysis server 10 of FIG. 1, and FIG. 3 is a diagram schematically showing the sub-configuration of the second box removal module 330 of the image analysis server 10 of FIG. 2. am.

Referring to FIGS. 2 and 3 , the image analysis server 10 may include a memory 200, a processor 300, and a communication module 400.

The processor 300 may be configured to process instructions of a computer program by performing basic arithmetic, logic, and input/output operations. Commands may be provided to the processor 300 from the memory 200 or the communication module 400. In addition, commands may be provided to the processor 300 through a communication channel between each component constituting the image analysis server 10.

The processor 300 forms a plurality of boxes for an object, maintains only the number of boxes corresponding to the object among the plurality of boxes, and deletes the remaining boxes. Data input/output, data processing, data management, communication network ( 40) It can perform various functions such as communication using . Specific components of the processor 300 for executing this will be described later.

Additionally, components of the processor 300 may include an artificial neural network previously trained through deep learning. For example, at least one of the components of the processor 300 may be an artificial neural network that implements RetinaNet, which will be described in detail later.

The memory 200 is a computer-readable recording medium and may include a non-permanent mass storage device such as RAM (Random Access Memory), ROM (Read Only Memory), and a disk drive.

The processor 300 may load the program code stored in the memory 200 and use it to count objects or determine the type of object. These program codes may be loaded from a recording medium (eg, DVD, memory card, etc.) readable by a separate computer, or may be transferred from another device through the communication module 400 and stored in the memory 200.

In addition, the memory 200 may be provided with a database 210 capable of storing data necessary to form a plurality of boxes for an object and delete the remaining boxes, leaving only the number of boxes corresponding to the object among the plurality of boxes. You can.

The communication module 400 may provide a function for the user terminal 20 and the image analysis server 10 or the administrator terminal 30 and the image analysis server 10 to communicate with each other through the communication network 40.

The image analysis server 10 includes a physical configuration of a box setting module 310, a first box removal module 320, a second box removal module 330, a pill count determination module 340, a counting module 350, and a type determination module 360. In addition, the second box removal module 330 may include a reference box setting unit 331, a set box setting unit 332, a comparison space setting unit 333, and a box removal unit 334 by pill coefficient comparison. A detailed explanation of this will be provided later.

FIG. 4 is a flowchart schematically showing an object counting method using an image analysis server executed by the object counting system 1 of FIG. 1, and FIG. 5 shows step S2 of steps S1 to S3 in FIG. 4 in more detail. It is a flowchart, and FIG. 6 is a flowchart showing step S36 among steps S32 to S36 of FIG. 5 in more detail, and FIG. 7 is a diagram conceptually showing receiving an object by the user terminal 20 shown in FIG. 1. , FIG. 8 is a diagram conceptually showing that RetinaNet, an object recognition deep learning model, is executed by the image analysis server 10 of FIG. 1 to form a plurality of boxes for one pill, and FIG. 9 is an image analysis of FIG. 1. This is a diagram conceptually showing a box formed in one pill (object) by executing NMS (non maximum suppression), an algorithm for removing boxes, by the server 10.

Referring to Figures 4 and 9, the object counting method using an image analysis server includes the step of receiving an image containing one or more objects by the user terminal 20 (S1); Forming a plurality of boxes for each object by the image analysis server 10, leaving only the number of boxes corresponding to the object among the plurality of boxes and deleting the remaining boxes (S2); It may include a step (S3) of counting the remaining boxes by the image analysis server 10 and transmitting the number corresponding to the boxes to the user terminal 20.

Objects included in one image that can be counted by the image analysis server 10 may include the same type of object with the same size and shape, or a plurality of types of objects with different sizes and shapes.

In this embodiment, the object included in one image counted by the image analysis server 10 is explained as an example of the same type of object.

In addition, below, the above-described process will be described in more detail, taking as an example that the object is a pill.

First, the step S1 of receiving an image containing one or more objects (eg, pills) by the user terminal 20 will be described in detail as follows.

The user can place the same type of pill with the same size and shape on the object plate 50 (see (a) of FIG. 7) and take an image of the pill through the user terminal 20 (see (a) of FIG. 7). see b)).

At this time, the pills must be placed on the object plate 50 so that they do not overlap.

However, the idea of the present invention is not limited to this, and the object counting system 1 may include a function to warn that pills are overlapped by the multi-analysis assistant 60 or the image analysis server 10, which will be described later. . A detailed explanation of this will be provided later.

The object plate 50 may be a flat plate on which a pill can be placed, and may be provided in a color that contrasts with the pill or in a color different from the pill. For example, if the pill is provided in white, the object plate 50 may be provided in black.

Thereafter, the image containing the pill captured by the user terminal 20 may be transmitted to the image analysis server 10.

In this embodiment, the user holds the user terminal 20 and takes an image as an example, but the spirit of the present invention is not limited to this and the user terminal 20 is used in the multi-analysis assistance device 60, which will be described later. You can also take an image by placing it (see FIG. 14), and a detailed explanation of this will be provided later.

Next, a plurality of boxes are formed for each object (e.g., a pill) by the image analysis server 10, and the remaining boxes are deleted, leaving only the number of boxes corresponding to the object among the plurality of boxes. The detailed description of the step (S2) is as follows.

The image analysis server 10 may receive an image containing a plurality of pills of the same type from the user terminal 20.

Afterwards, a plurality of boxes can be formed for one object by the box setting module 310 of the image analysis server 10 (S32).

For example, the box setting module 310 may be provided as an artificial neural network running RetinaNet, an object recognition deep learning model. When RetinaNet is running, multiple boxes can be created for each pill. However, the object recognition deep learning model that can be executed by the box setting module 310 is not limited to RetinaNet, and the box setting module 310 may include executing one or more CenterNet among YOLO.

When using RetinaNet, when learning a neural network in the method of detecting objects using boxes, the number of object samples is very small compared to the number of background samples, and the problems that arise due to this can be solved.

Specifically, RetinaNet may be an integrated network consisting of a backbone network and two task-specific subnetworks. The backbone network is responsible for calculating a convolutional feature map for the entire input image. The first subnet is a step that performs object classification convolutionally from the results of the backbone, and the second subnet can perform the role of convolutionally estimating the box (bounding box).

Figure 8 is a diagram conceptually showing that a plurality of boxes (B) are formed for one pill (object) by executing RetinaNet, an object recognition deep learning model, by the box setting module 310.

When RetinaNet is executed by the box setting module 310, if the pills are closely adjacent to each other, a plurality of boxes are formed for one pill, and the number of pills and the number of boxes are not the same. Therefore, in order to accurately count the number of pills even when the pills are closely attached, after RetinaNet is executed, a step must be taken to remove some of the plurality of boxes formed by RetinaNet.

Specifically, after RetinaNet is executed by the box setting module 310, an algorithm for removing some of a plurality of boxes formed in one object by the first box removal module 320 of the image analysis server 10 will be executed. (S34).

For example, the algorithm executed by the first box removal module 320 may be non maximum suppression (NMS). Here, NMS (non maximum suppression) can be understood as a non-maximum value suppression algorithm that, when comparing the current pixel with surrounding pixels, retains the maximum value and removes the other pixels.

Figure 9 conceptually shows a box formed in one pill (object) by executing NMS (non maximum suppression), an algorithm for removing boxes, by the first box removal module 320.

Even after NMS (non maximum suppression) is executed, if the pills are very close together, the number of pills and the number of boxes may be different.

For example, referring to Figure 9, it can be seen that five boxes (B1, B2, B3, B4, B5) are formed in three pills that are very closely attached. In this case, a step of leaving only one box for one object and deleting the remaining boxes may be performed in the second box removal module 330 (S36). Here, the second box removal module 330 may include a reference box setting unit 331, a set box setting unit 332, a comparison space setting unit 333, and a box removal unit 334 by pill coefficient comparison. And with this configuration, step S36 can be executed as follows (see FIG. 6).

Specifically, step S36 includes setting a random box among the remaining boxes as a reference box by the reference box setting unit 331 (S361); Setting a set box, which is a set of boxes overlapping the reference box, by the set box setting unit 332 (S362); A step of removing a space overlapping with the set box from the space occupied by the reference box by the comparison space setting unit 333 and setting the remaining space in the reference box as a comparison space (S363); If the ratio of the space occupied by the comparison space and the reference box is greater than the pill coefficient by the box removal unit 334 by pill coefficient comparison, a box set as the reference box is left, and the space occupied by the comparison space and the reference box is left. If the ratio is less than the pill coefficient, a step (S364) of removing the box set as the reference box may be included (see FIG. 5).

FIG. 10 is a conceptual diagram to help understand step S36 shown in FIG. 6.

With reference to FIGS. 1 to 10, step S36 is described as an example as follows.

When step S34 is executed by the first box removal module 320, a larger number of boxes than pills are formed for closely attached pills (for example, 5 boxes (B1 to B5) by 3 pills) ) can be formed.

In this case, the first box (B1), which is a random box among the five remaining boxes (B1 to B5), is set as the reference box, and the second box (B2) and the fourth box that overlap the first box (B1) are Box B4 and fifth box B5 are set as collection boxes.

Afterwards, the space overlapping with the set boxes (B2, B4, B5) is removed from the space occupied by the first box (B1), and the remaining space is set as the comparison space (C).

Afterwards, since the ratio of the space occupied by the comparison space (C) and the first box (B1), which is the reference box, is greater than the pill coefficient (comparison space (C)/space occupied by the reference box (B1) > pill coefficient), The first box B1 set as the reference box may remain.

Here, the pill coefficient represents the space where an object (pill) can exist, and can be set differently depending on the size and shape of the object (pill). The pill coefficient is greater than 0 and less than 1 (for example, the pill coefficient is 0.85 ) can be set to a value of

As such, the pill coefficient may be set by the pill coefficient determination module 340 of the image analysis server 10.

Specifically, the pill coefficient according to the size and shape of the object (pill) may be stored in the database 210, and an image including the object (pill) is transmitted from the user terminal 20 to the image analysis server 10. If so, the pill coefficient determination module 340 can match the pill coefficient stored in the database 210 according to the size and shape of the object (pill) and set the pill coefficient differently depending on the type of object. Conceptually, as the size of the pill increases, the pill coefficient can increase between 0 and 1.

Likewise, when the fourth box (B4) is set as the reference box, the ratio of the space occupied by the comparison space and the fourth box (B4), which is the reference box, is smaller than the pill coefficient, so the fourth box (B4) set as the reference box can be removed.

In this way, when going through steps S361 to S364, even if there are objects that are closely adjacent to each other, the number of objects and the number of boxes can be made the same.

Thereafter, a step (S3) of counting the remaining boxes by the image analysis server 10 and transmitting the number corresponding to the boxes to the user terminal 20 may be performed.

Specifically, the counting module 350 of the image analysis server 10 counts the remaining boxes and transmits them to the user terminal 20, and the user terminal 20 displays the counted number or sends a voice message to the user through a speaker. It can be passed on.

In addition, the type of object analyzed by the image analysis server 10 and the number of counted objects may be matched and stored in the database 210, and the user may check the type of object and the number of counted objects through the user terminal 20. You can also check the history of the number of .

Through this process, the user simply takes images of dozens to tens of thousands of pills and transmits them to the image analysis server 10, and the exact number of pills can be counted and notified to the user. It can reduce the time required for inventory research.

In addition, the above-described processor can be installed in the form of an application on the user terminal 20 or provided as a web page, and the user can download the application or access the web page and simply upload the image, and the pill included in the image The number can be automatically delivered to the user.

Hereinafter, the box setting module 310, the first box removal module 320, the second box removal module 330, the pill count determination module 340, and the counting module, which are sub-configurations of the image analysis server 10 described above. (350) is explained in more detail.

As described above, the box setting module 310 may execute an object recognition deep learning model to form a plurality of boxes for each object.

The first box removal module 320 may execute an algorithm to remove some of the plurality of boxes formed in each object.

The second box removal module 330 may leave only one box for one object and delete the remaining boxes.

Specifically, the second box removal module 330 may include a reference box setting unit 331, a set box setting unit 332, a comparison space setting unit 333, and a box removal unit 334 by pill coefficient comparison. You can,

The reference box setting unit 331 can set any box among the remaining boxes as the reference box.

The set box setting unit 332 can set a set box, which is a set of boxes that overlap a reference box.

The comparison space setting unit 333 may remove the space overlapping with the set box from the space occupied by the reference box and set the space remaining in the reference box as the comparison space.

If the ratio of the space occupied by the comparison space and the standard box is greater than the pill coefficient, the box removal unit 334 by pill coefficient comparison leaves a box set as the reference box, and the ratio of the space occupied by the comparison space and the standard box is the pill coefficient. If it is smaller, you can remove the box set as the reference box.

The pill coefficient determination module 340 may match the pill coefficient stored in the database 210 according to the size and shape of the object appearing in the image.

The counting module 350 can count the number of boxes corresponding to the object and transmit the number to the user terminal 20.

FIG. 11 is a diagram schematically showing a flowchart of transmitting a plurality of images to the image analysis server 10 using the object counting system 1 of FIG. 1 and counting one or more objects included in each of the plurality of images, FIG. 12 is a diagram showing a screen that appears on the user terminal 20 of FIG. 1 in the case of single analysis mode and multi-analysis mode, and FIG. 13 shows a plurality of images for which analysis has been completed by the image analysis server 10 in the case of multi-analysis mode. The number and type of objects included in each are shown on the screen of the user terminal 20 in FIG. 1.

In the object counting system 1 of the above-described embodiment, one image is transmitted to the image analysis server 10 through the user terminal 20, and a plurality of objects included in one image are analyzed by the image analysis server 10. Although described as an example, the following describes the object counting system 1 for an embodiment of transmitting a plurality of images to the image analysis server 10 and analyzing a plurality of objects included in each of the plurality of images. .

Before explaining the method of counting objects included in a plurality of images using an image analysis server, the screen of the user terminal 20 will be described with reference to FIGS. 12 and 13 as follows.

The screen of the user terminal 20 includes an image magnification unit 111, a single analysis button 112, a multi analysis button 113, an image input button 114, a multi analysis window 115, and a total number display unit 119. It can be included.

The image enlarger 111 may display an image being captured or captured by the user terminal 20.

A plurality of images captured by the user terminal 20 may be displayed in the multi-analysis window 115, and the number of objects for each image analyzed by the image analysis server 10 may be displayed.

In addition, the multi-analysis window 115 includes an image selection window 115a for selecting each image and a count display unit 115b for displaying the number of images for which analysis has been completed by the image analysis server 10. can be provided. Additionally, the multi-analysis window 115 may be provided with a delete button 116 that can delete each image.

The type display unit 118 can display the type of object included in the image selected by the image selection window 115a.

The total number display unit 119 may display the sum of objects included in all of the plurality of images displayed in the multi-analysis window 115.

Referring to FIGS. 11 to 13, the method of counting objects included in a plurality of images using an image analysis server uses a single analysis mode in which one image can be input by the user terminal 20 or a plurality of images can be input. A step in which a possible multi-analysis mode is selected (S10); When the multi-analysis mode is selected, a plurality of images containing one or more objects are input by the user terminal 20, and the plurality of input images are transmitted to the image analysis server 10 (S20); Counting the number of objects included in each of the plurality of images by the image analysis server 10 (S30); It may include a step (S40) in which the number of objects included in each of the plurality of images is displayed by the user terminal 20.

First, a detailed description of the step (S10) in which a single analysis mode in which one image can be input or a multi-analysis mode in which a plurality of images can be input is selected by the user terminal 20 is as follows.

The user can select a single analysis mode or a multi-analysis mode through the user terminal 20.

Specifically, the user can execute a single analysis mode by touching or clicking the single analysis button 112 displayed on the screen of the user terminal 20, and execute the multi-analysis mode by touching or clicking the multi-analysis button 113. You can do it.

Here, when the single analysis mode is selected, only one image is captured through the user terminal 20, and one image is transmitted to the image analysis server 10, so that only one image is analyzed. .

Additionally, when the multi-analysis mode is selected, it can be understood that a plurality of images are captured by the user terminal 20, the plurality of images are transmitted to the image analysis server 10, and all of the plurality of images are analyzed.

In addition, when the multi-analysis mode is selected, the user terminal 20 may be provided with an input window (not shown) for selecting the number of images to be captured. In this case, images as many as the number of images to be selected may be captured by the user. can be created.

For example, if 5 types of pills must be provided to patient A, the user can enter 5 in the input window, and if 5 images are input, the 5 images will be transmitted to the image analysis server 10. You can.

Next, when the multi-analysis mode is selected, a plurality of images containing one or more objects are input by the user terminal 20, and the plurality of input images are transmitted to the image analysis server 10 (S20) ) is explained in detail as follows.

When the user selects the multi-analysis mode, the multi-analysis window 115 is activated on the screen of the user terminal 20, and a plurality of captured images may be displayed in the multi-analysis window 115.

The user can edit multiple images displayed in the multi-analysis window 115. For example, the user can delete images that do not want to be analyzed by touching or clicking the delete button 116 of the multi-analysis window 115.

When a plurality of images containing one or more pills are input (photographed) by the user terminal 20, the user can input the type of pill shown in the images through the user terminal 20. However, the spirit of the present invention is not limited to this, and the type of pill may be automatically identified by the multi-analysis assistance device 60 and/or the image analysis server 10, which will be described later. A detailed explanation of this will be provided later.

The plurality of images input in this way can be transmitted to the image analysis server 10.

Next, a step (S30) in which the number of objects included in each of the plurality of images is counted by the image analysis server 10 will be described.

Specifically, in step S30, a plurality of boxes are formed for each object included in each of the plurality of images by the image analysis server 10, and the number of boxes corresponding to the object among the plurality of boxes formed in each image is Deleting the remaining boxes leaving only the box; It may include counting the number of boxes remaining in each of the plurality of images by the image analysis server 10 and transmitting the number corresponding to the boxes remaining in each of the plurality of images to the user terminal 20.

Here, the counting method of objects included in each image is the same as S2 and S3 described above, and the detailed description thereof will replace S2 and S3 described above.

Next, a step (S40) in which the number of objects included in each of the plurality of images is displayed by the user terminal 20 will be described.

Specifically, step S40 is a step in which a plurality of images are displayed in the multi-analysis window 115 of the user terminal 20: an object included in each of the plurality of images in the multi-analysis window 115 of the user terminal 20 A step in which the number of is displayed; Displaying the type of object included in each of the plurality of images on the type display unit 118 of the user terminal 20; It may include displaying the sum of objects included in all of the plurality of images on the total number display unit 119 of the user terminal 20 (see FIG. 13).

For example, four images are displayed in the multi-analysis window 115, and the number of pills is displayed on one side (eg, bottom) of each image.

In addition, a type display unit 118 is provided on one side of the multi-analysis window 115, and the type of the selected image (eg, Nexium tablets) may be displayed on the type display unit 118. At this time, the selected image among the plurality of images displayed in the multi-analysis window 115 may be displayed in a different color from the unselected image.

Meanwhile, the object counting system 1 of this embodiment may further include a multi-analysis assistance device 60 and a moving belt 70 for inputting a plurality of images in the multi-analysis mode of step S10 described above.

FIG. 14 is a diagram schematically showing the multi-analysis auxiliary device 60 on which the user terminal 20 of FIG. 1 can be mounted, and FIG. 15 shows the multi-analysis auxiliary device of FIG. 14 for easily performing the multi-analysis mode ( This is a diagram schematically showing the 60) and the moving belt 70.

Referring to FIGS. 14 and 15 , the object counting system 1 of this embodiment may further include a multi-analysis auxiliary device 60 and a moving belt 70 to easily perform the multi-analysis mode.

The multi-analysis assistance device 60 can be understood as a device on which the user terminal 20 can be seated, and the moving belt 70 can be understood as a device that can move a plurality of object plates 50.

When the multi-analysis auxiliary device 60 and the moving belt 70 shown in FIGS. 14 and 15 are provided, a plurality of images containing one or more objects are input by the user terminal 20 described above in S20. The steps can be easily realized.

Specifically, the step of inputting a plurality of images containing one or more objects by the user terminal 20 includes the step of seating the user terminal 20 on the terminal seating portion 67 of the multi-analysis assistance device 60. ; A step of seating a plurality of object plates 50 on which objects are placed on the moving belt 70; Step of sequentially positioning a plurality of object plates 50 at the bottom of the user terminal 20 as the moving belt 70 moves; And each object plate 50 stays at the bottom of the user terminal 20 for a certain period of time and then moves, so that the plurality of object plates 50 move at the bottom of the user terminal 20, and the user terminal 20 moves to each of the object plates 50. It may include generating a plurality of images by photographing an object placed on the object plate 50.

In addition, in the above-described embodiment, an embodiment in which only the number of objects of the same type is counted has been described. However, when using the object plate 50 including the type identification tag 52, the object counting system 1 may be used to count different types of objects. You can also judge the object of .

Specifically, a type identification tag 52 provided in one or more of letters, barcodes, and certain symbols may be provided on one side of the object plate 50. The type of object (pill) can be determined by this type identification tag 52.

For example, the user can place different types of pills on the object plate 50 according to the type identification tag 52 attached to the object plate 50, and the user terminal 20 determines the type of object plate 50. By photographing the identification tag 52, an image containing both the type identification tag 52 and the object is generated, or by creating an image containing the object and an image containing the type identification tag 52, respectively, and creating an image analysis server. By (10), the type and number of objects can be determined by matching and analyzing the type identification tag 52 and the object. At this time, the processor 300 of the image analysis server 10 may further include a type determination module 360 capable of determining the type identification tag 52.

In this case, after the step (S30) in which the number of objects included in each of the plurality of images is counted by the image analysis server 10, the type determination module 360 of the image analysis server 10 determines the object type. A step of determining the type of the object by matching the type identification tag 52; And a step of displaying the number and type of objects included in each of the plurality of images may be performed by the user terminal 20.

Specifically, data about the type of the object according to the type identification tag 52 is stored in the database 210, and the type determination module 360 receives data about the type of the object stored in the database 210 and determines the type of the object. You can judge the type.

For example, if the type identification tag 52 is provided with the symbol 1234 and the type of object corresponding to the symbol 1234 is stored in the database 210 as Nexium tablets, the user enters Nexium tablets on the object plate 50 displayed with the symbol 1234. By setting the definition, the type of object can be easily recognized by the image analysis server 10 without additional cumbersome work.

Below, a physical device capable of determining the type of the above-described object will be described in more detail.

The object counting system 1 according to an embodiment of the present invention includes an object plate 50 that provides a space for placing an object and includes a type identification tag 52 provided in one or more of letters, barcodes, and certain symbols. ; A user terminal 20 capable of photographing the object plate 50 to generate an image including one or more objects placed on the object plate 50 and an image including a type identification tag 52; and an image analysis server 10 capable of determining the number and type of objects included in the image. Here, the object and the type identification tag 52 may be photographed as one image or as separate images.

The object plate 50 includes a flat seating portion 55 on which an object can be placed; and a type identification tag 52 disposed outside the seating portion 55 and provided with one or more of letters, barcodes, and certain symbols. may include.

In addition, the object counting system 1 may further include a multi-analysis auxiliary device 60 including a terminal seating portion 67 on which the user terminal 20 can be placed, spaced apart from the object plate 50 at a preset distance. there is.

The multi-analysis auxiliary device 60 includes a lower portion 62 along which the object plate 50 moves; an upper surface portion 66 including a terminal seating portion 67 on which the user terminal 20 can be placed; And it may include a side part 64 connecting the lower part 62 and the upper surface part 66. Here, the height of the side portion 64 may be understood as the distance between the object plate 50 and the user terminal 20, and the side portion 64 may be provided with an adjustable height.

When using this multi-analysis assistant 60, an image can be created by placing the user terminal 20 on the terminal seating part 67 and photographing the object, making it easy to image the object placed on the object plate 50. You can take pictures easily.

Additionally, the multi-analysis assistant 60 may include a sensor 69 that can determine the overlap of objects placed on the object plate 50.

For example, the sensor 69 is provided on the side portion 64 of the multi-analysis assistant 60, the object plate 50 passes in front of the sensor 69, and the sensor 69 has the object plate 50. As it moves, the height of the object placed on the object plate 50 can be scanned. Here, the height of the object can be understood as the length measured in the vertical direction from the seating portion 55 of the object plate 50.

In other words, the image captured by the user terminal 20 can be understood as capturing one side (top surface) of the object, and the sensor 69 attached to the multi-analysis assistant 60 can capture the other side (side surface) of the object. ) can be understood as scanning.

As the object plate 50 passes in front of the sensor 69, the sensor 69 can scan all objects placed on the object plate 50 and a certain range of objects placed on the object plate 50. If more objects are scanned, the user can be notified.

At this time, the multi-analysis assistant 60 is provided with a speaker (not shown) connected to the sensor 69, which notifies the user through a warning sound or transmits a signal from the sensor 69 to the user terminal 20, A warning sound or warning display can be provided to the user through the terminal 20.

In this case, the user can check the objects placed on the object plate 50 and place them again without the objects overlapping.

In addition, the object counting system 1 further includes a moving belt 70 on which a plurality of object plates 50 are seated and capable of moving the plurality of object plates 50 to the bottom of the user terminal 20. can do.

Here, the moving belt 70 may be provided to form a closed curve. In this case, by placing a plurality of object plates 50 on the moving belt 70 forming a closed curve, the number of pills can be counted using the plurality of object plates 50.

Additionally, when a plurality of object plates 50 are provided, the colors of the seating portions 55 of the plurality of object plates 50 may be provided differently.

For example, the object plate 50 on which a red-colored object is placed may be provided in a green color, and the color of the seating portion 55 of the object plate 50 on which a white-colored object is placed may be provided in a black color. It can be. In this case, the image analysis server 10 can recognize the object more easily by distinguishing between the object and the background color.

The image analysis server 10 according to an embodiment of the present invention, the object counting system 1 including the same, an object counting method using an image analysis server, and a method of counting objects included in a plurality of images using an image analysis server are described above. Although described as a specific embodiment, this is merely an example and the present invention is not limited thereto, and should be construed as having the widest scope following the basic ideas disclosed in this specification. A person skilled in the art may combine and substitute the disclosed embodiments to implement embodiments not specified, but this also does not deviate from the scope of the present invention. In addition, a person skilled in the art can easily change or modify the embodiments disclosed based on the present specification, and it is clear that such changes or modifications also fall within the scope of the present invention.

1: Object counting system 10: Image analysis server
20: user terminal 30: administrator terminal
50: object plate 60: multi analysis aid
70: Moving belt

Claims (12)

Selecting a single analysis mode in which one image can be input or a multi-analysis mode in which a plurality of images can be input by the user terminal;
When a multi-analysis mode is selected, inputting a plurality of images containing one or more objects by the user terminal;
Transmitting a plurality of input images to an image analysis server;
Counting the number of objects included in each of the plurality of images by the image analysis server; and
A step of displaying the number of objects included in each of the plurality of images by the user terminal,
The step of counting the number of objects contained in each of the plurality of images by the image analysis server,
Forming a plurality of boxes for each object included in each of the plurality of images by the image analysis server, leaving only the number of boxes corresponding to the object among the plurality of boxes formed in each image and deleting the remaining boxes. ;
Counting the number of boxes remaining in each of the plurality of images by the image analysis server, and transmitting the number corresponding to the box remaining in each of the plurality of images to the user terminal,
The user terminal is,
An image enlarger where the image being shot can be displayed; and
A plurality of captured images can be displayed, the number of objects included in each analyzed image can be displayed, and a multi-analysis window can be used to edit the plurality of images,
When the single analysis mode is selected, only one image is captured by the user terminal and only one image is analyzed by the image analysis server,
When the multi-analysis mode is selected, a plurality of images are captured by the user terminal, a plurality of images are analyzed by the image analysis server, and the multi-analysis window is activated on the screen of the user terminal.
A method for counting objects included in multiple images using an image analysis server. According to claim 1,
The step in which the number of objects contained in each of a plurality of images is displayed by the user terminal,
Steps in which multiple images are displayed in the multi-analysis window of the user terminal:
Displaying the number of objects included in each of the plurality of images in the multi-analysis window of the user terminal;
Displaying the type of object included in each of the plurality of images on the type display of the user terminal; and
Including the step of displaying the sum of objects included in all of the plurality of images on the total number display of the user terminal.
A method for counting objects included in multiple images using an image analysis server. According to claim 1,
The step of inputting a plurality of images containing one or more objects by the user terminal,
A step in which the user terminal is seated on the terminal seating portion of the multi-analysis assistance device;
A step of seating a plurality of object plates on which objects are placed on a moving belt;
Step of sequentially positioning a plurality of object plates at the bottom of the user terminal as the moving belt moves; and
Each object plate stays at the bottom of the user terminal for a certain period of time and then moves, thereby moving the plurality of object plates to the bottom of the user terminal, and generating a plurality of images by photographing the object placed on each object plate by the user terminal. containing
A method for counting objects included in multiple images using an image analysis server. According to clause 3,
After the step in which the plurality of object plates on which the object is placed are seated on the moving belt,
The object plate is moved so that the object placed on the object plate passes through the front of the sensor of the multi-analysis aid.
A method for counting objects included in multiple images using an image analysis server. According to claim 1,
After the number of objects contained in each of the plurality of images is counted by the image analysis server,
Further comprising the step of determining the type of the object by matching the object and the type identification tag by the type determination module of the image analysis server.
A method for counting objects included in multiple images using an image analysis server.


delete delete delete delete delete delete delete

Images