JP6734546B2 - Information processing apparatus, control method thereof, and program - Google Patents

Description

The present invention relates to an information processing device, a control method therefor, and a program, and more particularly to a technique for notifying that a subject goes out of a facility or a subject is left on the basis of a shooting result by a camera. Is.

2. Description of the Related Art In recent years, there has been known a mechanism of using a network camera to detect and notify that a child has become lost in a facility where, for example, a parent (user) and a child (another user led by the user) are present. There is.

In Patent Document 1, for example, the face of the leader photographed by the surveillance camera, the current location of the leader identified by the photographed surveillance camera, the face of the child photographed by the surveillance camera, It is disclosed that if the taken surveillance camera is away from the present location of the identified child for a certain period of time, it is determined that the child is lost and the notification is given.

JP, 2007-60528, A

An object of the present invention is to provide a mechanism for notifying that a subject goes out of the facility or that the subject is left on the basis of the shooting result by the camera.

According to the present invention, area information indicating an area in which image data in which the first object is specified is specified by specifying a first object included in each image data obtained by the camera capturing each area. And a second subject that is registered as a subject related to the first subject and that is included in each image data obtained by the camera capturing each area. Then, the first object is a facility using the area information managed by the managing unit that manages the area information indicating the area where the image data in which the second object is specified is captured. outlet, or in a state of being taken at other areas of the facility is not a outlet near the area, determining means for determining the second object is photographed by the facility outlet or outlets around the area, by the determination means, the first object of the facility outlet or in a state of being taken at other areas of the facility is not a outlet near the area, the second object of the facility outlet, or near the outlet, on condition that it is determined to have been taken with the area, the notification for notifying that the second object will get out of the facility, or in left behind the first object within the facility Notification control means for controlling so as to give a notification for notifying that the user is present.

Further, the present invention is the control method, wherein the management means specifies the first subject included in each image data obtained by the camera photographing each area, and the first subject is specified. The second object registered as a subject related to the first subject and managing the area information indicating the area in which the captured image data was captured, and each obtained by the camera capturing each area. A management step of specifying the second subject included in the image data and managing area information indicating an area in which the image data in which the second object is specified is photographed, and a determination means are performed by the management step. using a managed area information, the first subject of the facility outlet or in a state of being taken at other areas of the facility is not a outlet near the area, the second object of the facility outlet, or a determination step of determining taken with the outlet near the area, the notification means, by the determination step, the first subject of the facility outlet, or in other areas of not the facility at the exit near the area shooting state, the second object of the facility outlet, or on condition that it is determined to have been taken at the exit near the area, said second object will get out of the facility And a notification control step of controlling to perform a notification for notifying or a notification for notifying that the first subject is left at the facility .

Also, the present invention causes a computer, characterized by a program to function as the aforementioned information processing apparatus.

According to the present invention, it is possible to give a notification regarding the subject going out of the facility or leaving the subject on the basis of the shooting result by the camera.

It is a figure showing composition of an information processing system concerning an embodiment of the present invention. FIG. 2 is a block diagram showing a hardware configuration of an information processing device applicable to the server 101 and the mobile terminal 102 shown in FIG. 1. It is a flow chart which shows processing which server 101 concerning an embodiment of the present invention performs. It is a figure which shows an example of the flowchart which shows the detailed process of the lost child alert process performed by step S304 shown in FIG. It is a figure which shows an example of the flowchart which shows the detailed process of the lost child alert process performed by step S304 shown in FIG. It is a figure which shows an example of the flowchart which shows the detailed process of the alerting process near an exit performed by step S304 shown in FIG. It is a figure which shows an example of the flowchart which shows the detailed process of the child leaving alert process performed by step S304 shown in FIG. It is a figure which shows an example of a management table. It is a figure which shows the other example of a management table. It is a figure which shows an example of the notification content which the server 101 notifies. It is a figure which shows an example of the correspondence table with which the identification information of a network camera and the area name of the area where the said network camera is arrange|positioned were matched.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

Hereinafter, a preferred embodiment to which the present invention is applied will be described in detail with reference to the accompanying drawings.

FIG. 1 is a diagram showing a configuration of an information processing system according to an embodiment of the present invention.

Reference numeral 101 shown in FIG. 1 denotes a server, which is communicably connected to each of the network cameras 103 to 114 via a wired or wireless network 114.

The server 101 is connected to the mobile terminal 102 via a wireless network so that they can communicate with each other.

The server 101 is an application example of an information processing apparatus that receives image data captured by a camera connected to a network via the network, identifies the lost child using the image data, and notifies the lost child. ..

The network cameras 103 to 114 are imaging devices (cameras) that are provided in the same facility and can be connected to the network 114. Each of the network cameras 103 to 114 is a camera capable of shooting a moving image and/or a still image. Hereinafter, moving images and/or still images are also referred to as images.

Each of the network cameras 103 to 114 has a function of transmitting captured video data to the server 101.

The network cameras 103 to 106 are cameras installed in the area A in the facility, and each of the network cameras 103 to 106 stores identification information for identifying the network camera (individual) in the memory. ..

The network cameras 107 to 110 are cameras installed in the area B in the facility, and each of the network cameras 107 to 110 stores identification information for identifying the network camera (individual) in the memory. ing.

The network cameras 111 to 114 are cameras installed in the area C of the facility, and each of the network cameras 111 to 114 stores identification information for identifying the network camera (individual) in the memory. ing. Area C is an exit area or an area near the exit.

Each of the network cameras 103 to 114 photographs a parent 115 (guardian) and a child 117 in the facility as subjects. A color code indicating identification information for identifying the parent (individual) is attached to the parent 115 (guardian). For example, the clothes of the parent 115 (guardian) are provided with the color code sticker. Further, a color code indicating identification information for identifying the child (individual) is attached to the child 117 of the parent. For example, the clothes of the child 117 are attached with the seal of the color code.

That is, each of the network cameras 103 to 114 photographs the color code of the parent 115 (guardian) in the facility and the color code of the child 117 as a subject.

Although the color code is used here for identifying the individual, any other code such as a barcode or an image of the face of the individual may be used as long as the individual can be identified.
The mobile terminal 102 is a mobile terminal carried by the parent 115.

In FIG. 1, it has been described that the areas in which the network cameras are installed are areas A to C. However, in addition to the areas A to C, the area D to Z are also provided with network cameras in the same manner. ..

The hardware configuration of the information processing apparatus applicable to the server 101 and the mobile terminal 102 shown in FIG. 1 will be described below with reference to FIG.

FIG. 2 is a block diagram showing a hardware configuration of an information processing apparatus applicable to the server 101 and the mobile terminal 102 shown in FIG.

In FIG. 2, reference numeral 201 denotes a CPU, which centrally controls each device and controller connected to the system bus 204. Further, the ROM 203 or the external memory 211 is necessary to realize a control program of the CPU 201, such as a BIOS (Basic Input/Output System), an operating system program (hereinafter, OS), and a function executed by each server or each PC. Various programs to be described later are stored.

A RAM 202 functions as a main memory and a work area of the CPU 201. The CPU 201 loads various programs necessary for executing the processing from the ROM 203 or the external memory 211 into the RAM 202 and executes the loaded programs to realize various operations.

An input controller 205 controls input from an input device 209 such as a keyboard (KB) or a pointing device such as a mouse (not shown). A video controller 206 controls display on a display device 210 such as a liquid crystal display. Note that the display device 210 is a touch panel and also functions as an input device that can be operated by pressing the screen on the display.

A memory controller 207 is installed in an external storage device (hard disk (HD)) for storing a boot program, various applications, font data, user files, edit files, various data, a flexible disk (FD), or a PCMCIA card slot. It controls access to an external memory 211 such as a CompactFlash (registered trademark) memory connected via an adapter.

A communication I/F controller 208 connects and communicates with an external device via a network, and executes communication control processing on the network. For example, communication using TCP/IP is possible.

The CPU 201 enables display on the display device 210, for example, by executing outline font rasterization processing in the display information area in the RAM 202. Further, the CPU 201 enables a user instruction with a mouse cursor or the like (not shown) on the display device 210.

Various programs to be described later for implementing the present invention are recorded in the external memory 211 and are executed by the CPU 201 by being loaded into the RAM 202 as necessary. Furthermore, the definition file and various information tables used when the above program is executed are also stored in the external memory 211, and a detailed description thereof will be described later.
<Explanation of FIG. 3>

FIG. 3 is a flowchart showing processing executed by the server 101 according to the embodiment of the present invention.

The processing of each step shown in steps S301 to S305 is executed by the CPU 201 of the server 101 under the control of the program recorded in the external memory 211 of the server 101.

First, each of the network cameras (103 to 114) sequentially transmits the captured video data to the server 101. In the example here, the data of the captured image is transmitted at predetermined time intervals (for example, 1 minute).

Further, here, each network camera stores identification information for identifying each network camera (individual) in the memory, so that each network camera stores not only video data but also each memory in the memory. The identification information that exists is transmitted to the server 101.

Then, the server 101 receives, from each network camera, video data from each network camera and identification information for identifying the network camera (S301).

Here, the video data includes not only still image data but also moving image data. The data of the frame image forming the moving image data can be the target of the processing described below, as with the still image data.

Step S301 is an application example of an acquisition unit that sequentially acquires, from each camera, image data (video data) continuously captured by each camera over time.

Then, the server 101 analyzes the video data received in S301 and acquires identification information for identifying the subject (S302). For example, the server 101 identifies the color code attached to the subject from the received video data, and reads and analyzes the identified color code to identify the subject (color information). The code ID (identification information for identifying an individual) is acquired. The user who has been photographed can be specified by acquiring this identification information.

That is, step S301 is an application example of the user identification unit that identifies the user photographed in the image data by analyzing the image data acquired by the acquisition unit.

Next, the server 101 identifies the area of the network camera (area where the subject is) according to the identification information of the network camera received in step S301 and the correspondence table shown in FIG.
<Description of FIG. 11>

FIG. 11 is a diagram showing an example of a correspondence table in which the identification information of the network camera and the area name of the area in which the network camera is arranged are associated with each other.

In the table of FIG. 11, area information (area name) indicating the area where the camera is installed is stored in the storage unit (memory) for each camera.

In the correspondence table shown in FIG. 11, the identification information (001, 002, 003,...) Of each network camera and the area name (area A, area B, area C) of the area where each network camera is installed. ,...) are associated with each other and registered in advance in the memory.

For example, the area in which the network camera whose network camera identification information is 001 is installed is Area A.

In step S303 of FIG. 3, the area where the network camera is installed is acquired by acquiring the area name stored in the correspondence table (FIG. 11) corresponding to the identification information of the network camera received in step S301. Identify.

That is, step S303 is based on the area information (FIG. 11) indicating the area in which the camera, which has acquired the image data for which the user has been specified in step S302 and has been acquired in step S301, is located in the area where the user is specified in step S302. It is an application example of the area specifying means for specifying.

Then, the server 101, the area name of the area specified here, the identification information (color code ID (identification information for identifying an individual)) for identifying the subject acquired in step S302, and the current The time (current time) is associated with each other and registered in the management table shown in FIG. 8 or FIG. 9 (S303).

In step S303, the area information indicating the area where the user (parent) identified in step S302 is located identified in step S303 and the user identified in step S302 identified in step S303 and led to by the user. It is an application example of a storage unit that stores area information indicating an area where another user (child) is present in a memory (storage unit) for each shooting time when the image data is shot by the camera.
<Explanation of FIGS. 8 and 9>
FIG. 8 is a diagram showing an example of the management table.
FIG. 9 is a diagram showing another example of the management table.

The management tables shown in FIGS. 8 and 9 are configured by associating items of “ID”, “family ID”, “guardian flag”, “notification destination”, and “time”.

The “ID” is identification information (color code ID) (for example, 100) for identifying the subject.

The "family ID" is an ID (for example, 1) for identifying a group such as a family. Here, the subjects (people) in which the same value is stored indicate that they are the same family (group).

The “guardian flag” is a flag indicating whether or not the user is a guardian. Here, "1" indicates that the person is a guardian, and "0" indicates that the person is not a guardian (for example, a child).

The “notification destination” stores information of the notification destination. Here, as the notification destination, for example, a mail address of an electronic mail that can be delivered to the guardian's mobile terminal 102 (for example, aaa@a.ne.jp) is stored.

The “time” item includes time (..., “10:55”, “10:56”, “10:57”, “10:58”, “10:” for each predetermined time (for example, 1 minute). 59” and “11:00” (current time)) are provided as items, and at each time, the area name of the area where the subject identified by the “ID” (for example, 100) was stored is stored. There is.

In FIG. 8, the subject (person) whose “ID” is 100 will be described as an example. In FIG. 8, the subject (person) whose “ID” is 100 is the same family (person) as the subject (person) whose “ID” is 101. Group) and a guardian (the subject (person) whose “ID” is 101 is not a guardian (child)), and the notification destination is aaa@a. ne. jp, being in area F at "10:55", being in area E at "10:56", being in area D at "10:57", being in area C at "10:58". It means that it was in the area B at "10:59" and that it was in the area A at "11:00" (current time).

The “ID”, “family ID”, “guardian flag”, “notification destination”, and “time” shown in FIG. 8 or 9 are registered in advance, and in step S303, “ID” acquired in step S302. The area name of the area specified in step S303 is registered in the column in which the identification information (color code ID) for identifying the subject is associated with the current time (current time).

In this way, the server 101 generates the management table shown in FIG. 8 or 9 by updating and registering the management table shown in FIG. 8 or 9 every predetermined period (for example, 1 minute). ..

Next, the server 101 executes the lost child alert processing shown in FIG. 4 or the lost child alert processing shown in FIG. 5 (step S304). Further, in step S304, the server 101 processes the exit neighborhood alert processing shown in FIG. 6 and the child leaving alert processing shown in FIG. 7 in parallel.

That is, the server 101 processes the lost child alert processing shown in FIG. 4, the lost child alert processing shown in FIG. 5, the exit neighborhood alert processing shown in FIG. 6, and the child leaving alert processing shown in FIG. 7 in parallel.

Then, when the server 101 executes the process of step S304, it determines whether or not the administrator (user) has received an end instruction, and if it is determined that the end instruction has been received (YES), the process ends. If it is determined that the end instruction has not been received (NO), the process returns to step S301.
<Description of FIG. 4>

Next, the detailed process of the lost child alert process executed in step S304 shown in FIG. 3 will be described with reference to FIG.

FIG. 4 is a diagram showing an example of a flowchart showing detailed processing of the lost child alert processing executed in step S304 shown in FIG.

The processing of each step shown in steps S401 to S404 is executed by the CPU 201 of the server 101 under the control of the program recorded in the external memory 211 of the server 101.
The flowchart shown in FIG. 4 is repeatedly executed for each same family ID.

First, the server 101 executes the process shown in FIG. 4 in order from the top record in the upper part of the management table (FIG. 8 or FIG. 9) for each same family ID.

First, the server 101 extracts from the management table (FIG. 8 or FIG. 9) data of “ID”, “family ID”, “guardian flag”, and “notification destination” among records having the same family ID, The area name at the current time and the area name for a predetermined time (for example, 6 seconds here) from the current time are acquired (step S401).
For example, a record in which the "family ID" is 1 will be described as an example.

In step S401, of the records having "ID" of 100 and 101, information having "ID" of 100 ("ID" is 100, "family ID" is 1, "guardian flag" is 1, and "notification destination" is aaa@a.ne.jp, the area when "10:55" was: area F, the area when "10:56" was area E, the area when "10:57" was area: D, Area: Area C when "10:58", Area: Area B when "10:59", Area: Area A when "11:00", and "ID" of 101 Information (“ID” is 101, “Family ID” is 1, “Guardian flag” is 0, “Notification destination” is NULL, Area: Area F when “10:55”: “10:56” Area that was present: area E, area that was present at "10:57": area D, area that was present at "10:58": area C, area that was present at "10:59": area B, Area B) of "11:00" is acquired.

Then, the server 101 determines whether or not the areas in which the current parent and child are present are different according to the area name of the parent and child at the current time acquired in step S401 (S402).

For example, the area name at the current time (11:00) in which the guardian flag is set to 1 and the area name at the current time (11:00) in which the parent flag is set to 0 are compared and are the same. If so, it is determined that the parent and child are in the same area (NO). On the other hand, if they are not the same, it is determined that the parent and child are not in the same area (the areas where the current parent and child are different are different) (YES).

If it is determined in step S402 that the parent and child are in the same area (NO), the server 101 moves the process to step S305, and if it is determined that the parent and child are in different areas (YES), The process proceeds to step S403.

Then, the server 101 has the parent and child for the latest past predetermined period (for example, 2 seconds) among the parent and child area names for a predetermined time (for example, 6 seconds) from the current time acquired in step S401. It is determined whether the areas are different (step S403).

For example, in step S403, the area name at the time (10:58) when the parent flag is set to 1 matches the area name at the time (10:58) when the parent flag is set to 0. And the area name at the time (10:59) when the parent flag is set to 1 matches the area name at the time (10:59) when the parent flag is set to 0. Determine whether or not.

In the server 101, the area name at the time (10:58) when the parent flag is set to 1 matches the area name at the time (10:58) when the parent flag is set to 0. Moreover, if the area name at the time (10:59) when the parent flag is set to 1 and the area name at the time (10:59) when the parent flag is set to 0 match. If it is determined, it is determined that the areas where parents and children are the same in the latest past predetermined period (for example, 2 seconds) are not different (the areas where parents and children are the same) (NO), while the guardian flag is set to 1 The area name at the designated time (10:58) does not match the area name at the designated guardian flag (0:58), or the guardian flag is set to 1. If it is determined that the area name at the time (10:59) currently set and the area name at the time (10:59) when the guardian flag is set to 0 do not match, the most recent past It is determined that areas where parents and children are present for a predetermined period (for example, 2 seconds) are different (YES).

Steps S402 and S403 are past from the last photographing time (for example, 11:00 (current) in FIG. 8) when the area information specified by step S303 (area specifying means) is stored in the memory (storage means). Area information (area name) indicating the area where the parent (user) is stored and area information (area name) indicating the area where the child (other user) is stored, stored in step S303 (storage means) within a predetermined time. It is an application example of the determination means for determining whether or not each is different.

If the server 101 determines in step S403 that the areas of the parent and child for the latest past predetermined period (for example, 2 seconds) are different (YES), the parent 101 indicates that the child is lost with the same family ID. The notification destination (for example, aaa@a.ne.jp) whose flag is set to 1 is notified.
An example of the content notified here is shown in 1001 of FIG.

For example, when the family ID in FIG. 8 is 1, the travel routes of the guardian (parent) (guardian flag is 1) and the child (guardian flag is 0) are the same from 10:55 to 10:59. However, at 11:00, the areas with parents and children are different.

Even if the area where the parent and the child are different at the current time in this way, since the area where the parent and the child are the same within the predetermined period immediately after the current time, it is determined as NO in S403, Parents are not notified that their child is lost.

Therefore, for example, even when the parent happens to move to an area different from that of the child at the timing of updating the management table, it is possible to reduce erroneous determination that the child is a lost child.

By performing the process shown in FIG. 4 in this way, for example, when the color code cannot be recognized temporarily due to an obstacle, or the parent and the child are moving together, it happens that the parent and the child respectively Even if it is recognized that the user is in a different area, it is possible to reduce the false determination of a lost child.

Further, for example, as in the case of the family ID 4 in FIG. 8, when the color code cannot happen to be imaged due to an obstacle or the like at present and the area where the parent or child is present cannot be recognized (when the area currently is NULL is present). ), even if the parent is not notified that the child is lost, it is possible to reduce false determination that the child is lost.

In this way, when the lost child alert processing shown in FIG. 4 is executed using the management table shown in FIG. 8 as an example, the parents IDs 1 and 4 are not notified to the parent that the child is lost, but the family ID In case of 2 or 3, the parent is notified that the child is lost.

However, if the walking speed of the child is slower than the walking speed of the parent and it happens that there is a parent in the area ahead of the area where the child is, as in the case where the family ID in the management table shown in FIG. Area may be detected.

That is, in the case of the process of FIG. 4, even if the parent-child route is the same, the parent is notified that the child is lost even if the parent-child area at each time is different. A mechanism for reducing such erroneous determination will be described later with reference to FIG.
<Description of FIG. 10>
FIG. 10 is a diagram showing an example of notification contents notified by the server 101.

In 1001 of FIG. 10, together with the fact that the child is lost, the position of the network camera where the child is currently photographed is notified as the current location of the child.

If the server 101 determines in step S403 that the areas of the parent and child in the latest past predetermined period (for example, 2 seconds) match (NO), the process proceeds to processing step S305.

The server 101 can also execute the lost child alert processing shown in FIG. 5 instead of the lost child alert processing shown in FIG.
<Explanation of FIG. 5>

Next, the detailed process of the lost child alert process executed in step S304 shown in FIG. 3 will be described with reference to FIG.

FIG. 5 is a diagram showing an example of a flowchart showing detailed processing of the lost child alert processing executed in step S304 shown in FIG.

The processing of each step shown in steps S401 to S404, S405, and S406 shown in FIG. 5 is executed by the CPU 201 of the server 101 under the control of the program recorded in the external memory 211 of the server 101.
The flowchart shown in FIG. 5 is repeatedly executed for each same family ID.

First, the server 101 executes the process shown in FIG. 5 for each same family ID in order from the top record in the upper part of the management table (FIG. 8 or FIG. 9).
In FIG. 5, steps that perform the same processing as in FIG. 4 are assigned the same reference numerals.
Therefore, in FIG. 5, only processes different from those in FIG. 4 will be described.

The flowchart shown in FIG. 5 differs from the flowchart of FIG. 4 only in that step S405 and step S406 are added.

When it is determined in step S403 that the areas where the parent and child are in the latest past predetermined period (for example, 2 seconds) are different in step S403 (step S403: YES), the server 101 uses the parent and child times acquired in step S401. Based on the area name, the moving route for a predetermined time (for example, 5 seconds) for each parent and child is specified (step S405).

In step S405, area information (area name) indicating the area in which the parent (user) is stored and area information (area name) in which the child (other user) is stored, stored in step S303 (storage unit), ( The area information (area name) specified by step S303 (area specifying means) according to FIG. 8 or FIG. 9) is stored in the memory (storage means) by step S304 (storage means) at the last photographing time (for example, FIG. 8 is an application example of a specifying unit that specifies a moving path of a parent (user) and a moving path of a child (other user) within a predetermined time in the past from 11:00 (current) of 8).
Here, a specific example of step S405 will be described.

For example, when the record having the family ID 2 shown in FIG. 8 (the record having the color code ID of 102 and the record having the color code ID of 103) is acquired and processed in step S401, the color code ID is The subject (person) 102 is a movement route of area F (10:56), area E (10:57), area D (10:58), area C (10:59), and area B (11:00). The subject (person) whose color code ID is 103 is moving in area F (10:55), area E (10:56), area D (10:57), and area C (10: 58) and area B (10:59). In other words, the subject (person: parent) whose color code ID is 102 and the subject (person: child) whose color code ID is 103 are moving on the same movement path with a shift of 1 minute.

In this way, in step S405, it is possible to specify the movement route for each parent and child for a predetermined time (for example, 5 seconds) from the area name acquired in step S401 at each time of the parent and child.

Next, the server 101 determines whether or not the movement paths of the parent and child for a predetermined time (for example, 5 seconds) specified in step S405 are different (step S406), and when it is determined that they are different (YES). ), and notifies that the child is lost (step S404). On the other hand, if it is determined that the parent and child have the same travel route for a predetermined time (for example, 5 seconds) (NO), the process proceeds to step S305 without notifying that the child is lost. Since step S404 is the same process as step S404 of FIG. 4, description thereof will be omitted here.

Step S406 is an application example of the movement route determination unit that determines whether or not the movement route of the parent (user) identified by Step S405 (identification unit) and the movement route of the child (other user) are different. ..

In step S404, the area information (area name) indicating the area where the parent (user) is present and the area information (area name) indicating the area where the child (other user) is present are respectively obtained by steps S402 and S403 (determining means). When it is determined that they are different, and it is determined by step S406 (moving route determination means) that the moving route of the parent (user) and the moving route of the child (other user) are different (step S406: YES). The server 101 controls to notify the terminal operated by the parent or the administrator that the child (other user) is lost (control means).

Further, in step S402 and step S403 (determination means), the area information (area name) indicating the area where the parent (user) is present matches the area information (area name) indicating the area where the child (other user) is present. (Step S402: NO, Step S403: NO) or Steps S402 and S403 (determining means) are used to determine the area information (area name) indicating the area where the parent (user) is and the area where the child (other user) is. Even when it is determined that the area information (area name) indicated is different from each other, the movement route of the parent (user) and the movement route of the child (other user) are determined by step S406 (movement route determination means). When it is determined that the two match (step S406: NO), the server 101 controls not to notify that the child (other user) is lost (control means).

In addition, in step 404, the server 101 stores that the child (other user) is lost and the area information (area name) of the other user is stored in the memory (storage unit) in step S303 (storage unit). The area information (area name) of the other user (child) at the last shooting time (for example, 11:00 (current) in FIG. 9) is notified.

As described above, in addition to the above-described effects of the process shown in FIG. 4, the process shown in FIG. 5 actually causes the parent and child to move together, but happens to be continuous when the parent and the child are in different areas. Even if it is recognized as such, it is possible to reduce an erroneous determination of a lost child.
<Description of FIG. 6>

Next, detailed processing of the exit vicinity alert processing executed in step S304 shown in FIG. 3 will be described with reference to FIG.

FIG. 6 is a diagram showing an example of a flowchart showing detailed processing of the exit vicinity alert processing executed in step S304 shown in FIG.

The processing of each step shown in steps S601 to S609 is executed by the CPU 201 of the server 101 under the control of the program recorded in the external memory 211 of the server 101.
The flowchart shown in FIG. 6 is repeatedly executed for each same family ID.

First, the server 101 executes the process shown in FIG. 6 in order from the top record in the upper part of the management table (FIG. 8 or FIG. 9) for each same family ID.

First, as in step S401, the server 101 selects “ID”, “family ID”, “guardian flag”, “notification destination” from the management table (FIG. 8 or FIG. 9) among records having the same family ID. Data, the area name at the current time, and the area name for a predetermined time (for example, 6 seconds in this case) from the current time are acquired (step S601).

Then, the server 101 determines whether the areas where the current parent and child are present are different according to the area names of the parent and child at the current time acquired in step S601 (S602).

Since the process of step S602 is the same as the process of step S402 described above, description using a specific example is omitted here.

If it is determined in step S602 that the parent and child are in the same area (NO), the server 101 moves the process to step S305, and if it is determined that the parent and child are in different areas (YES), The process proceeds to step S603.

Then, the server 101 has the parent and child for the latest past predetermined period (for example, 2 seconds) among the area names of the parent and child for a predetermined time (here, for example 6 seconds) from the current time acquired in step S601. It is determined whether the areas are different (step S603).

Since the process of step S603 is the same as the process of step S403 described above, the description using a specific example is omitted here.

If the server 101 determines in step S603 that the areas of the parent and child in the latest past predetermined period (for example, 2 seconds) are different (YES), the process proceeds to step S604. On the other hand, if it is determined that the areas of the parent and child in the latest past predetermined period (for example, 2 seconds) are not different (the same) (NO), the process proceeds to step S608.

When it is determined in step S603 that the areas where the parent and child are in the latest past predetermined period (for example, 2 seconds) are different in step S603 (step S603: YES), the server 101 determines that the parent and child are acquired at step S601. Based on the area name that has been set, a moving route for a predetermined time (for example, 5 seconds) for each parent and child is specified (step S604).

Since step S604 is the same process as step S405 described above, description using a specific example is omitted here.

Then, the server 101 determines whether or not the movement paths of the parent and child for a predetermined time (for example, 5 seconds) specified in step S604 are different (step S605), and if it is determined that they are different (YES). The process proceeds to step S606. On the other hand, if it is determined that the parent and child have the same movement route for a predetermined time (for example, 5 seconds) (NO), the process proceeds to step S608.

Since step S605 is the same process as step S406 in FIG. 5, description using a specific example is omitted here.

In step S606, the server 101 determines from the area name acquired in step S601 whether the area where the parent is currently is not the exit area or the area near the exit but the child is in the exit area or the area near the exit. ..

For example, when the record having the family ID of 3 (the record having the ID of 104 and the record having the ID of 105) shown in the management table 900 of FIG. 9 is the processing target, the determination process of step S606 is performed. In this case, at 11:00 (current), the parent (subject with ID 104) is in area A, and the child (subject with ID 105) is in area C (exit area or near exit area). I understand.

Therefore, in the case where the family ID shown in the management table 900 of FIG. 9 is a record of 3, in step S606, the area where the parents are is not the exit area or the exit vicinity area, but the child is in the exit area or the exit vicinity area ( Step S606: YES).

When the server 101 determines from the area name acquired in step S601 that the area where the parent is currently is not the exit area or the area near the exit, the child is in the exit area or the area near the exit (step S606: (YES), Notify the notification destination of the record that the guardian flag of the record to be currently processed is set to 1 (step) S607).
An example of the content notified in step S607 is shown in 1002 of FIG.

Then, when the server 101 executes the process of step S607, the process proceeds to step S305.

In step S606, the server 101 determines from the area name acquired in step S601 that the area where the parent is currently is the exit area or the area near the exit, or that the child is not in the exit area or the area near the exit. In that case (step S606: NO), the process proceeds to step S305.

In step S608, the server 101 determines from the area name acquired in step S601 whether the area where the parent is currently is not the exit area or the area near the exit, and the child is in the exit area or the area near the exit. ..

Since the process of step S608 is also similar to the process of step S606, description using a specific example is omitted here.

Then, in step S<b>608, the server 101 determines from the area name acquired in step S<b>601 that the current parent area is not the exit area or the exit vicinity area, and the child is in the exit area or the exit vicinity area. If (YES), the fact that the child may go out of the facility is notified to the notification destination of the record in which the guardian flag of the record currently being processed is set to 1 (step S609).

An example of the content notified in step S609 is shown in 1003 of FIG. Then, when the server 101 executes the process of step S607, the process proceeds to step S305.

In addition, in step S608, the server 101 determines that the area currently having a parent is the exit area or the exit vicinity area or the child is not in the exit area or the exit vicinity area based on the area name acquired in step S601. If it is determined that (NO), the process proceeds to step S305.

In steps S606 and S608, the area information (area name) of the parent (user) at the last shooting time (for example, 11:00 (current) in FIGS. 8 and 9) in the table of FIG. 8 or 9 is displayed. Whether or not the area information (area name) of the child (other user) is the area information (area C) indicating the exit area or the area near the exit, not the area information (area C) indicating the exit area or the area near the exit. It is an application example of the exit determination means for determining.

Then, in step S607 and step S609, the server 101 notifies that another user will leave the facility.

By executing the process shown in FIG. 6, if the area where the parent is not the exit area and the area where the child is is the exit area, an alert is sent to the guardian. For example, the child goes out of the facility. It is possible to prevent the search by the color code from being disabled.

Furthermore, if the child has come to the exit area in a lost state (YES in S602, YES in S603, YES in S605, YES in S606), the child is lost in step S607. In order to notify the guardian that there is one and that the child will go out of the facility, the lost child will go out of the facility while the parent is inside the facility, making it impossible to search by color code Can be prevented.

If the parent and the child are moving together, but it happens that the parent and the child are in different areas (NO in S604 and S605), the notification that the child is lost is not issued in S607. Therefore, it is possible to prevent erroneous determination of a lost child.

Further, even if the parent and the child are moving together, but it happens that it is recognized that the parent and the child are in different areas (NO in S604 and S605), the parent is not in the exit area in step S608. If it is determined that the child is in the exit area (YES), in order to notify that the child will leave the facility, for example, the child was not lost, but the child suddenly headed for the exit. Even if the child runs away from the parent and the child tries to get out of the facility from the exit (because the parent cannot grasp the child's position), the parent can know that the child is in the exit area and go outside the facility. It is possible to prevent children from leaving.
<Description of FIG. 7>

Next, the detailed processing of the child leaving alert processing executed in step S304 shown in FIG. 3 will be described with reference to FIG.

FIG. 7 is a diagram showing an example of a flowchart showing detailed processing of the child leaving alert processing executed in step S304 shown in FIG.

The processing of each step shown in steps S701 to S707 is executed by the CPU 201 of the server 101 under the control of the program recorded in the external memory 211 of the server 101.
The flowchart shown in FIG. 7 is repeatedly executed for each same family ID.

First, the server 101 executes the process shown in FIG. 7 in order from the top record in the upper part of the management table (FIG. 8 or FIG. 9) for each same family ID.

First, as in step S401, the server 101 selects “ID”, “family ID”, “guardian flag”, “notification destination” from the management table (FIG. 8 or FIG. 9) among records having the same family ID. Data, the area name of the current time, and the area name of a predetermined time (for example, 6 seconds) from the current time are acquired (step S701).

Since the process of step S701 is the same as the process of step S401 described above, description using a specific example is omitted here.

Then, the server 101 determines whether or not the areas in which the current parent and child are present are different according to the area name of the parent and child at the current time acquired in step S701 (S702).

Since the process of step S702 is the same as the process of step S402 described above, description using a specific example is omitted here.

If it is determined in step S702 that the parent and child are in the same area (NO), the server 101 moves the process to step S305, and if it is determined that the parent and child are in different areas (YES), The process proceeds to step S703.

Then, the server 101 has the parent and child for the latest past predetermined period (for example, 2 seconds) among the parent and child area names for a predetermined time (for example, 6 seconds) from the current time acquired in step S701. It is determined whether the areas are different (step S703).

If the server 101 determines in step S703 that the areas in which the parent and child for the latest past predetermined period (for example, 2 seconds) are different are different (YES), the process proceeds to step S704, and the latest past If it is determined that the areas where the parent and child are for a predetermined period (for example, 2 seconds) are not different (they are the same) (NO), the process proceeds to step S305.

Since the process of step S703 is also the same as that of step S403 described above, the description using a specific example is omitted here.

Next, the server 101 determines whether or not it is possible to recognize the area where the parent is present for a predetermined period (for example, 2 seconds) of the present time and the latest past (step S704). Specifically, it is determined whether or not the area having the parent for the latest past predetermined period (for example, 2 seconds) among the area names acquired in step S701 is NULL. NULL is a value registered in the management table (FIG. 8 or 9) when the ID cannot be read from the color code.

When it is determined that the server 101 has been able to recognize the area where the parent is present (for example, 2 seconds) for the current period and the latest past predetermined period (for example, 2 seconds) (NULL) (YES), the process proceeds to step S705. On the other hand, if it is determined that the area where the parent is present for the current period and the latest past predetermined period (for example, 2 seconds) can be recognized (NO), the process proceeds to step S305.

Next, the server 101 determines whether or not the area where the child is currently is the area within the facility (step S705).

Specifically, by determining whether or not the current area of the record in which the guardian flag acquired in step S701 is 0 is NULL, it is determined whether or not the area currently having children is the area in the facility. Determine whether.

When the server 101 determines that the area where the child is present is the area inside the facility (the current area of the record with the guardian flag 0 acquired in step S701 is not NULL) (YES), If the process proceeds to step S706 and it is determined that the area currently having children is not the area within the facility (the current area of the record with the parental flag of 0 acquired in step S701 is NULL) The process proceeds to step S305.

In step S706, the server 101 determines whether the latest parent area detected in the past is the exit area. Specifically, in the data of the record with the guardian flag of 1 acquired in step S701, the area name in which the area where the parent is detected is the area C (exit area) at the closest time (closest) from the present time. Determine whether there is.

In step S<b>706, the server 101 determines that the latest parent area detected in the past is the exit area (the area name in which the area with the parent is detected is the area C (exit area at the closest time (closest) from the present). )) (step S706: YES), the notification destination of the record having the guardian flag of 1 among the records acquired in step S701, indicating that the child is left in the facility. To (step S707).
An example of the content notified in step S707 is shown in 1004 of FIG.

Steps S704, S705, and S706 are area information (area information indicating an area where a parent (user) is present within a predetermined time in the past from the last shooting time (for example, 11:00 (current)) in FIGS. 8 and 9. Name) is NULL that does not indicate the area in the facility, and the area information (area name) that indicates the area with the child (other user) at the last shooting time indicates the area (other than NULL) in the facility. Area information (area name) indicates the area (other than NULL) in the facility Area information (area name) of the parent (user) at the last shooting time indicates the exit area or the area near the exit in the facility It is an application example of the leaving judgment means for judging whether or not it is information (area C).

Then, in step S707, the fact that the child (other user) is left in the facility is notified.

As described above, by executing the process shown in FIG. 7, the area where the parent is present is NULL and the area where the child is present is the area in the facility at the present time and the most recent past period, and the most recent past When the area of the parent detected in step 1 is the exit area, the parent can be notified that the child has been left behind in the facility.

Therefore, for example, if the child is left inside the facility and the parent goes out of the facility, the parent can understand that the child was left inside the facility and the color code can be displayed by the obstacle. Even if the child is not recognized temporarily, or the parent and child are moving together, but even if the parent happens to go out of the facility first and the child goes out of the facility later, It is possible to reduce the judgment that the user has left it.

As described above, according to the present invention, although the user and the other user led by the user are moving together, it is continuously determined that the user and the other user led by the user are in different areas. Even in the case of being erroneous, it is possible to reduce erroneous determination that other users have become lost.

Further, according to the present invention, it is possible to prevent a child from leaving the facility alone and leaving the facility, and a parent from leaving the child in the facility and leaving the facility.

Further, an object of the present invention is to supply a storage medium storing a program code of software that realizes the functions of the above-described embodiments to a system or apparatus, and the computer (or CPU or MPU) of the system or apparatus stores the storage medium. It is needless to say that it is achieved by reading and executing the program code stored in.

In this case, the program code itself read from the storage medium realizes the functions of the above-described embodiments, and the program code itself and the storage medium storing the program code constitute the present invention.

As the storage medium for supplying the program code, for example, a flexible disk, a hard disk, an optical disk, a magneto-optical disk, a CD-ROM, a CD-R, a magnetic tape, a non-volatile card, a ROM or the like can be used.

Further, by executing the program code read by the computer, not only the functions of the above-described embodiments are realized, but also the OS (basic system or operating system) running on the computer based on the instruction of the program code. It is needless to say that this also includes a case where the above) performs a part or all of the actual processing, and the processing realizes the functions of the above-described embodiments. An information processing device (computer) capable of reading and executing this program code realizes the functions of the above-described embodiments.

Furthermore, after the program code read from the storage medium is written in the memory provided in the function expansion board inserted in the computer or the function expansion unit connected to the computer, the function is executed based on the instruction of the program code. It goes without saying that a case where a CPU or the like included in an expansion board or a function expansion unit performs a part or all of actual processing and the functions of the above-described embodiments are realized by the processing is also included.

101 server 102 mobile terminal 103 network camera 104 network camera 105 network camera 106 network camera 107 network camera 108 network camera 109 network camera 110 network camera 111 network camera 112 network camera 113 network camera 114 network camera 115 parent (subject)
116 color code 117 child (subject)
118 color code 119 network

Claims (4)

The camera identifies the first subject included in each image data obtained by photographing each area, and manages the area information indicating the area in which the image data in which the first subject is identified is photographed, Further, the second subject, which is a second subject registered as a subject related to the first subject and is included in each image data obtained by photographing each area by the camera, is specified, Management means for managing area information indicating an area where the image data in which the second subject is specified is captured;
Using said area information managed by the management means, the first object of the facility outlet, or in a state of being taken at other areas of the facility is not a outlet near the area, the second object is the determination means for determining shot facility outlet, or outlets around the area,
By the determination means, the first object of the facility outlet or in a state of being taken at other areas of the facility is not a outlet near the area, the second object of the facility outlet, or near the outlet, on condition that it is determined to have been taken with the area, the notification for notifying that the second object will get out of the facility, or in left behind the first object within the facility Notification control means for controlling to perform notification to notify that there is
An information processing apparatus comprising: The information processing apparatus according to claim 1 , wherein the first subject and the second subject are persons. Area information indicating an area in which the image data in which the first subject is specified is specified by the management unit specifying the first subject included in each image data obtained by the camera photographing each area. And a second subject registered as a subject related to the first subject, the second subject being included in each image data obtained by the camera capturing each area. And a management step of managing area information indicating an area in which the image data in which the second subject is specified is photographed,
Determination means, by using the area information managed by the management step, the first subject of the facility outlet, or in a state of being taken at other areas of the facility is not a outlet near the area, the second or a a determination process subject is photographed by the facility outlet or outlets around the area, the,
Notifying means, by the determination step, the first subject of the facility outlet or in a state of being taken at other areas of the facility is not a outlet near the area, the second object of the facility outlet, or on condition that it is determined to have been taken at the exit near the area, the notification for notifying that the second object will get out of the facility, or the first object to the facility A notification control step of controlling to give a notification for notifying that it is left behind ,
A control method comprising: The computer program for functioning as an information processing apparatus according to claim 1 or 2.

Images