WO2020250544A1 - 遠隔監視装置、遠隔監視方法、コンピュータプログラム及び記録媒体 - Google Patents

遠隔監視装置、遠隔監視方法、コンピュータプログラム及び記録媒体 Download PDF

Info

Publication number
WO2020250544A1
WO2020250544A1 PCT/JP2020/014123 JP2020014123W WO2020250544A1 WO 2020250544 A1 WO2020250544 A1 WO 2020250544A1 JP 2020014123 W JP2020014123 W JP 2020014123W WO 2020250544 A1 WO2020250544 A1 WO 2020250544A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
user terminal
time
remote monitoring
images
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2020/014123
Other languages
English (en)
French (fr)
Japanese (ja)
Inventor
裕司 田原
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
NEC Corp
Original Assignee
NEC Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by NEC Corp filed Critical NEC Corp
Priority to JP2021525922A priority Critical patent/JP7533458B2/ja
Priority to US17/617,040 priority patent/US20220230441A1/en
Publication of WO2020250544A1 publication Critical patent/WO2020250544A1/ja
Anticipated expiration legal-status Critical
Priority to US18/237,714 priority patent/US20230401856A1/en
Priority to US18/237,766 priority patent/US20230401857A1/en
Priority to US18/237,778 priority patent/US20230401858A1/en
Priority to US18/659,140 priority patent/US20240290103A1/en
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
    • G06F16/58—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
    • G06F16/5866—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using information manually generated, e.g. tags, keywords, comments, manually generated location and time information
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q30/00—Commerce
    • G06Q30/06—Buying, selling or leasing transactions
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G—PHYSICS
    • G08—SIGNALLING
    • G08B—SIGNALLING SYSTEMS, e.g. PERSONAL CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
    • G08B25/00—Alarm systems in which the location of the alarm condition is signalled to a central station, e.g. fire or police telegraphic systems
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
    • H04N7/188—Capturing isolated or intermittent images triggered by the occurrence of a predetermined event, e.g. an object reaching a predetermined position
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/103—Static body considered as a whole, e.g. static pedestrian or occupant recognition

Definitions

  • the present invention relates to the technical fields of remote monitoring devices, remote monitoring methods, computer programs and recording media.
  • Patent Document 1 As a device of this type, for example, a device for monitoring the display area of a store has been proposed (see Patent Document 1). Other related techniques include Patent Documents 2 and 3.
  • the present invention has been made in view of the above problems, and an object of the present invention is to provide a remote monitoring device, a remote monitoring method, a computer program, and a recording medium capable of appropriately grasping the state of a monitored object. And.
  • One aspect of the remote monitoring device of the present invention is a storage means for storing a plurality of images to which position information and time information are added, and a distribution in which the first image among the plurality of images is distributed to a user terminal.
  • the distribution means is attached to the first image when (i) a first switching instruction indicating image switching in one direction of the first image is received from the user terminal.
  • Position information indicating a position different from the position indicated by the first position information which is the position information is given, and the same time as the time indicated by the first time information which is the time information given to the first image.
  • the second image to which the time information indicating the above is added is distributed to the user terminal, and (ii) the second image indicating the image switching from the user terminal to the other direction intersecting with the one direction of the first image.
  • position information indicating the same position as the position indicated by the first position information is given, and time information indicating a time different from the time indicated by the first time information is added.
  • the third image is delivered to the user terminal.
  • One aspect of the remote monitoring method of the present invention is a storage means for storing a plurality of images to which position information and time information are added, and a distribution means for delivering the first image among the plurality of images to a user terminal.
  • This is a remote monitoring method in a remote monitoring device including, and is given to the first image when a first switching instruction indicating image switching in one direction of the first image is received from the user terminal.
  • Position information indicating a position different from the position indicated by the first position information which is the given position information is given, and is the same as the time indicated by the first time information which is the time information given to the first image.
  • the position information indicating the same position as the position indicated by the first position information is given, and the time information indicating a time different from the time indicated by the first time information is given.
  • One aspect of the computer program of the present invention causes a computer to execute one aspect of the remote monitoring method described above.
  • One aspect of the recording medium of the present invention is a recording medium on which one aspect of the computer program described above is recorded.
  • the observer can appropriately grasp the state of the monitoring target.
  • FIG. 1 is a diagram showing an outline of a remote monitoring system according to an embodiment.
  • the remote monitoring system 1 includes a remote monitoring device 10 installed in a monitoring center, user terminals 20 and 30, and a plurality of monitoring cameras C1 to C8 installed in a store to be monitored. ing.
  • the arrangement and number of the surveillance cameras C1 to C8 in FIG. 1 are examples, and are not limited thereto.
  • shelves 1 to 16 are installed as product display shelves in the stores to be monitored, but the present invention is not limited to these.
  • the number of stores to be monitored may be two or more.
  • the remote monitoring device 10, the user terminal devices 20 and 30, and the monitoring cameras C1 to C8 are connected via a network (not shown) such as the Internet.
  • the video signals output from each of the surveillance cameras C1 to C8 are sequentially transmitted to the remote monitoring device 10 via the network.
  • the remote monitoring device 10 includes various sensors installed in the store to be monitored (for example, a temperature sensor for refrigerating or freezing equipment, a remaining amount detection sensor for beverage dispensers, a vibration sensor for preventing theft of cash registers, etc.) (shown in the figure).
  • the sensor signal output from (1) is acquired via the network.
  • FIG. 2 is a block diagram showing a hardware configuration of the remote monitoring device 10 according to the embodiment.
  • the remote monitoring device 10 includes a CPU (Central Processing Unit) 11, a RAM (Random Access Memory) 12, a ROM (Read Only Memory) 13, a storage device 14, an input device 15, and an output device 16.
  • the CPU 11, RAM 12, ROM 13, storage device 14, input device 15, and output device 16 are connected to each other via a data bus 17.
  • the CPU 11 reads a computer program.
  • the CPU 11 may read a computer program stored in at least one of the RAM 12, the ROM 13, and the storage device 14.
  • the CPU 11 may read a computer program stored in a computer-readable recording medium using a recording medium reading device (not shown).
  • the CPU 11 may acquire (that is, may read) a computer program from a device (not shown) arranged outside the remote monitoring device 10 via a network interface.
  • the CPU 11 controls the RAM 12, the storage device 14, the input device 15, and the output device 16 by executing the read computer program.
  • a logical functional block for remotely monitoring the store to be monitored is realized in the CPU 11. That is, the CPU 11 can function as a controller for remote monitoring of the store.
  • the configuration of the functional block realized in the CPU 11 will be described in detail later with reference to FIG.
  • the RAM 12 temporarily stores the computer program executed by the CPU 11.
  • the RAM 12 temporarily stores data temporarily used by the CPU 11 when the CPU 11 is executing a computer program.
  • the RAM 12 may be, for example, a D-RAM (Dynamic RAM).
  • the ROM 13 stores a computer program executed by the CPU 11.
  • the ROM 13 may also store fixed data.
  • the ROM 13 may be, for example, a P-ROM (Programmable ROM).
  • the storage device 14 stores data stored by the remote monitoring device 10 for a long period of time.
  • the storage device 14 may operate as a temporary storage device of the CPU 11.
  • the storage device 14 may include, for example, at least one of a hard disk device, a magneto-optical disk device, an SSD (Solid State Drive), and a disk array device.
  • the input device 15 is a device that receives an input instruction from the user of the remote monitoring device 10.
  • the input device 15 may include, for example, at least one of a keyboard, a mouse and a touch panel.
  • the output device 16 is a device that outputs information about the remote monitoring device 10 to the outside.
  • the output device 16 may be a display device capable of displaying information about the remote monitoring device 10.
  • FIG. 3 is a block diagram showing a functional block realized in the CPU 11 of the remote monitoring device 10.
  • an image acquisition unit 111, an image processing unit 112, a communication unit 113, a detection unit 114, and a notification unit 115 are realized as logical functional blocks in the CPU 11.
  • the image acquisition unit 111 receives the video signals output from each of the surveillance cameras C1 to C8, and acquires a plurality of time-continuous frame images (that is, camera images) for each surveillance camera (that is, camera images). Step S101).
  • the image processing unit 112 performs predetermined image processing such as distortion correction processing and human erasing processing on the frame image. After that, the image processing unit 112 extracts a predetermined image area from the frame image and generates an extracted image. The image processing unit 112 stores the extracted image in the storage device 14 (step S102). Here, time information (typically, time information based on the time indicated by the time stamp of the frame image) and position information corresponding to the extracted image area are added to the extracted image.
  • time information typically, time information based on the time indicated by the time stamp of the frame image
  • position information corresponding to the extracted image area are added to the extracted image.
  • the image processing unit 112 may store the extracted image extracted from a part of the frame images (for example, the frame image at 00 minutes per hour) acquired in the process of step S101 in the storage device 14. .. In this case, the image processing unit 112 does not have to target all the frame images acquired in the process of step S101 to the predetermined image processing. Further, since existing techniques can be applied to the distortion correction process and the human erasing process, detailed description thereof will be omitted.
  • a predetermined image area to be extracted from the frame image is set so that an image for each product display shelf is generated as the extracted image, for example, as shown in FIG.
  • a predetermined image area may be set manually or automatically, for example, when the remote monitoring system 1 is introduced.
  • the communication unit 113 determines whether or not there is an image request from the user terminal (here, for convenience, the user terminal 20) (step S201). If it is determined in the process of step S201 that there is no image request (step S201: No), the operation is terminated. Then, after a predetermined time (for example, several tens of milliseconds to several hundreds of milliseconds) has elapsed, the process of step S201 may be performed again.
  • a predetermined time for example, several tens of milliseconds to several hundreds of milliseconds
  • the communication unit 113 delivers the initial image to the user terminal 20 (step S202).
  • the initial image is provided with time information indicating, for example, the time closest to the current time among the plurality of extracted images stored in the storage device 14, and corresponds to, for example, shelf 1 (see FIG. 1). It may be an extracted image to which the position information to be used is added.
  • the communication unit 113 determines whether or not there is an image switching instruction from the user terminal 20 (step S203). When it is determined in the process of step S203 that there is no image switching instruction (step S203: No), the communication unit 113 determines whether or not there is an end instruction from the user terminal 20 (step S204).
  • step S204: Yes If it is determined in the process of step S204 that there is an end instruction (step S204: Yes), the operation is terminated. On the other hand, if it is determined in the process of step S204 that there is no end instruction (step S204: No), the process of step S203 is performed (that is, the standby state is set until there is an image switching instruction or an end instruction).
  • step S203 When it is determined in the process of step S203 that there is an image switching instruction (step S203: Yes), the communication unit 113 is in one direction in the user terminal 20 (in the present embodiment, as an example of one direction, the left-right direction). It is determined whether or not it is an image switching instruction to (explain an example of adopting) (step S205).
  • step S205 When it is determined in the process of step S205 that the image switching instruction is in the left-right direction (step S205: Yes), the communication unit 113 is based on the time information given to the image currently distributed to the user terminal 20. An extracted image to which time information indicating the same time as the indicated time is added and position information indicating a position different from the position indicated by the position information assigned to the currently distributed image is added. , Delivered to the user terminal 20 (step S206).
  • the image shown in FIG. 6A When, for example, the image shown in FIG. 6A is currently delivered to the user terminal 20, the image shown in FIG. 6B or FIG. 6C is delivered to the user terminal 20 by the process of step S206. It will be.
  • the time shown at the bottom of each of the images of FIGS. 6A to 6C is the time indicated by the time information given to each image.
  • step S205 when it is determined that the image switching instruction is not in the left-right direction (step S205: No), the image switching instruction is given in another direction on the user terminal 20 (that is, in a direction intersecting in one direction). Therefore, in the present embodiment, as an example of another direction, an image switching instruction to (explain an example in which the vertical direction is adopted) is given.
  • the communication unit 113 is provided with time information indicating a time different from the time indicated by the time information given to the image currently distributed to the user terminal 20, and the image currently distributed.
  • the extracted image to which the position information indicating the same position as the position indicated by the position information given to the user terminal 20 is given is delivered to the user terminal 20 (step S207).
  • the image shown in FIG. 7A When, for example, the image shown in FIG. 7A is currently delivered to the user terminal 20, the image shown in, for example, FIG. 7B or FIG. 7C is delivered to the user terminal 20 by the process of step S207. It will be.
  • the time shown at the bottom of each of the images of FIGS. 7A to 7C is the time indicated by the time information given to each image.
  • the above-mentioned “left-right direction image switching instruction” and “up-down direction image switching instruction” are “swipe left-right direction” and “up-down direction”, respectively. It may be “swipe” or “tap at the right or left end of the tablet terminal” and “tap at the top or bottom of the tablet terminal”.
  • the above-mentioned “image switching instruction in the left-right direction” and “image switching instruction in the up-down direction” are “instructions for switching images in the vertical direction", respectively, at the right end or left end of the displayed image. It may be “click” and “click on the upper end or the lower end of the displayed image”.
  • the communication unit 113 is given time information indicating the same time as the time indicated by the time information given to the image currently distributed to the user terminal 20, and is currently distributed. It is desirable to deliver the extracted image to the user terminal 20 to which the position information indicating the position adjacent to the position indicated by the position information given to the image is added.
  • the user terminal 20 is a tablet terminal, and the above-mentioned "image switching instruction in the left-right direction” is a "swipe in the left-right direction”.
  • the distribution unit 111 receives each time “swipe from right to left” is input. , "Image corresponding to shelf 1, image corresponding to shelf 2, image corresponding to shelf 3, ..., image corresponding to shelf 15, image corresponding to shelf 16", and images are delivered to the user terminal 20 in this order. (See FIG. 1).
  • the distribution unit 111 receives each time “swipe from left to right” is input. Images are distributed to the user terminal 20 in the order of "image corresponding to shelf 16, image corresponding to shelf 15, image corresponding to shelf 14, ..., image corresponding to shelf 2, image corresponding to shelf 1". Good. That is, when the direction of the image switching instruction is reversed, the order in which the images are switched may also be reversed.
  • the user of the user terminal 20 can view the images delivered to the user terminal 20 at the same intervals as when actually patrolling the store.
  • the "time indicated by the time information given to the image” is not limited to the actual time when the image was generated (for example, captured by the surveillance camera C1 or the like), and the expression mode of the time information. It is a concept including the time represented by. Therefore, the "image to which the time information indicating the same time as the time indicated by the time information given to the currently distributed image is added" is the actual time when the currently distributed image is generated. It is not limited to images generated at exactly the same time.
  • time information is represented by "hours and minutes” (that is, when seconds or less of the actual time is truncated), for example, an image generated at "11:00:00".
  • time indicated by the time information of the image generated at "11:00:20" is “11:00”
  • both are regarded as images generated at the same time in the remote monitoring system 1.
  • the time information is represented by "5 minute increments”, for example, both the image generated at "11:00” and the image generated at "11:02" are the times indicated by the time information. Is "11:00", so that the remote monitoring system 1 treats both as images generated at the same time.
  • the "position indicated by the position information given to the image” refers to the actual position (for example, latitude, longitude and point) of the range (or the point representing the range) shown in the image. It is a concept that includes not only the position represented by the altitude) but also the position represented by the expression style of the position information. Therefore, the "image to which the position information indicating the same position as the position indicated by the position information given to the currently distributed image” is given is the same range as the range shown in the currently distributed image. It is not limited to the captured image.
  • the position information is represented by the "code in the store sketch" (for example, “shelf 1", “shelf 2", ... In FIG. 1)
  • the position information given to each of the two images is If they show the same code, both are treated as images at the same position in the remote monitoring system 1.
  • the detection unit 114 assigns two extracted images to which position information indicating the same position is given and time information indicating different times is added. By comparison, at least one of foreign matter, abnormality and missing item in the store is detected.
  • the detection unit 114 further detects an abnormality based on sensor signals output from various sensors installed in the store to be monitored.
  • the detection unit 114 detects the shelf allocation from the image showing the product display shelf among the plurality of extracted images stored in the storage device 14. Since existing technologies can be applied to each of foreign matter detection, abnormality detection, shortage detection, and shelf allocation detection, detailed description thereof will be omitted.
  • the notification unit 115 issues an alarm to at least one of the user terminals 20 and 30 when at least one of a foreign object, an abnormality, and a missing item is detected by the detection unit 114.
  • the notification unit 115 issues an alarm and displays an image showing the status of at least one of the detected foreign matter and the abnormality on at least one of the user terminals 20 and 30. May be sent to.
  • the notification unit 115 compares the plurality of images with each other according to a predetermined comparison viewpoint. Then, the notification unit 115 may transmit one image selected based on the comparison result to at least one of the user terminals 20 and 30.
  • Examples of the above “comparative viewpoint” include "position and size in the image of an image area showing a foreign substance or anomaly”, “image resolution”, “presence or absence of an obstacle that hides a part of the foreign substance or anomaly", and the like. Can be mentioned.
  • the “communication unit 113", “storage device 14", and “detection unit 14" correspond to examples of “delivery means”, “storage means”, and “detection means” in the appendix described later, respectively.
  • the “image switching instruction in the left-right direction” and the “image switching instruction in the up-down direction” correspond to examples of the “first switching instruction” and the “second switching instruction” in the appendix described later, respectively.
  • the remote monitoring system 1 when an image switching instruction is given in the left-right direction, for example, time information indicating the same time as the time indicated by the time information given to the image currently distributed to the user terminal 20. Is given, and the extracted image to which the position information indicating the position different from the position indicated by the position information given to the currently delivered image is given is delivered to the user terminal 20. Therefore, the user of the user terminal 20 can change the position in the store where the fixed point observation is performed by issuing the image switching instruction in the left-right direction.
  • a monitor for example, each user of the user terminals 20 and 30
  • a monitor can appropriately grasp the state of the store as a monitoring target and its time change.
  • the image processing unit 112 includes (i) an extracted image stored in the storage device 14 and given position information indicating one position, and (ii) a frame image after the frame image on which the extracted image is based. Yes, and the subject included in the extracted image is compared with another frame image containing the same subject. Then, the image processing unit 112 may generate an extracted image from the other frame image on condition that the amount of change in the state of the subject is larger than a predetermined amount.
  • the "predetermined amount” is a value that determines whether or not to generate an extracted image from another frame image, and may be set in advance as a fixed value or as a variable value according to some parameter. For example, when the subject is a product display shelf, the "predetermined amount” may be set based on the degree of decrease in the displayed products.
  • the "image acquisition unit 111" and the “image processing unit 112" are examples of the “acquisition means” and the “extraction means” in the appendix described later, respectively.
  • the user terminals 20 and 30 may have a function of processing (for example, text input, drawing, etc.) the image delivered from the remote monitoring device 10. For example, suppose that the user of the user terminal 20 performs the processing as shown in FIG. 9 on the image distributed from the remote monitoring device 10.
  • the communication unit 113 of the remote monitoring device 10 that has received the processed image that specifies the user terminal 30 as the destination from the user terminal 20 may transfer the processed image to the user terminal 30.
  • the "communication unit 113" corresponds to an example of the "transfer means" in the appendix described later.
  • the remote monitoring device 10 targets a plurality of stores.
  • the position information given to each of the plurality of extracted images stored in the storage device 14 may include store information related to the store (for example, identification information related to the store).
  • attribute information indicating an attribute related to the subject may be added to each of the plurality of extracted images. Examples of attributes include beverages, fresh food, cooked rice, bread, frozen foods, confectionery, magazines, books, visitors, and the like.
  • the communication unit 113 of the remote monitoring device 10 has the same time as the time indicated by the time information given to the image currently distributed to the user terminal 20. Time information indicating the time is given, and location information including store information related to a store different from the store related to the store information included in the location information given to the currently distributed image is given.
  • the extracted image may be distributed to the user terminal 20.
  • the communication unit 113 uses the extracted image as the extracted image to which the attribute information indicating the same attribute as the attribute information indicated by the attribute information currently distributed to the user terminal 20 is added. It is desirable to deliver to the terminal 20.
  • the remote monitoring system 1 is not limited to monitoring stores, but is, for example, monitoring shopping streets, parks, parking lots, outdoor parts of schools, etc. using one or more surveillance cameras installed outdoors, and is installed indoors. It can also be applied to using one or more surveillance cameras, for example, monitoring common areas of buildings such as entrances, monitoring inside factories, and the like.
  • the remote monitoring device 10 determines whether or not there is an image switching instruction from the user terminal 20, and when it is determined that there is an image switching instruction, the image switching instruction is given.
  • the extracted images selected accordingly are distributed to the user terminal 20.
  • the user terminal 20 may determine whether or not there is an image switching instruction, and if it is determined that there is an image switching instruction, display an extracted image selected in response to the image switching instruction.
  • the user terminal 20 has the same hardware configuration as the remote monitoring device 10 (see FIG. 2), and determines whether or not there is an image switching instruction in the CPU of the user terminal 20 to instruct the image switching.
  • a functional block for example, a functional block corresponding to the communication unit 113 or the like described above for displaying an extracted image selected in response to an image switching instruction when it is determined that there is an image may be realized.
  • the user terminal 20 determines whether or not the image switching instruction is input to the user terminal 20 via the input device (device corresponding to the input device 15 in FIG. 2) included in the user terminal 20 (that is, the user terminal 20 It may be determined whether or not there is an image switching instruction by determining (whether or not the provided CPU has acquired the image switching instruction).
  • the extracted image may be stored in the storage device of the user terminal 20 (the device corresponding to the storage device 14 in FIG. 2).
  • a second embodiment according to the remote monitoring system 1 will be described.
  • the second embodiment is the same as the first embodiment described above, except that the operation of the remote monitoring device 100 is partially different. Therefore, with respect to the second embodiment, the description overlapping with the first embodiment will be omitted, and the common parts on the drawings will be indicated by the same reference numerals, and basically different points will be described.
  • the image processing unit 112 performs person detection processing and person recognition processing as predetermined image processing on a plurality of frame images acquired from the surveillance cameras C1 to C8.
  • the image processing unit 112 sets identification information (hereinafter, appropriately referred to as "person identification information") for the detected person (for example, a clerk, a visitor, etc.).
  • person identification information for the detected person (for example, a clerk, a visitor, etc.).
  • the image processing unit 112 extracts a predetermined image range including the detected person from the frame image in which the person is detected, and generates the extracted image.
  • the image processing unit 112 adds the person identification information related to the detected person to the extracted image. Therefore, the person identification information related to the same person is given to each of the plurality of extracted images in which the same person is shown.
  • the communication unit 113 indicates time information indicating the same time as the time indicated by the time information given to the image currently distributed to the user terminal 20. Is given, and the extracted image to which the person identification information different from the person identification information given to the currently delivered image is given is delivered to the user terminal 20.
  • the communication unit 113 indicates a time different from the time indicated by the time information given to the image currently distributed to the user terminal 20.
  • the extracted image to which the information is given and the same person identification information as the person identification information given to the currently distributed image is given is distributed to the user terminal 20.
  • the remote monitoring device includes a storage means for storing a plurality of images to which position information and time information are added, and a distribution means for delivering the first image among the plurality of images to a user terminal.
  • the distribution means has (i) a position assigned to the first image when it receives a first switching instruction indicating image switching in one direction of the first image from the user terminal. Position information indicating a position different from the position indicated by the first position information which is information is given, and the same time as the time indicated by the first time information which is the time information given to the first image is indicated.
  • a second image to which time information is added is delivered to the user terminal, and (ii) a second switching instruction indicating image switching from the user terminal to another direction intersecting the one direction of the first image.
  • position information indicating the same position as the position indicated by the first position information is given, and time information indicating a time different from the time indicated by the first time information is added.
  • It is a remote monitoring device characterized by delivering 3 images to the user terminal.
  • the remote monitoring device indicates, as the second image, a position adjacent to the position indicated by the first position information when the distribution means receives the first switching instruction from the user terminal.
  • the remote monitoring device according to Appendix 1, wherein an image to which position information is given and time information indicating the same time as the time indicated by the first time information is given is delivered to the user terminal. Is.
  • each of the plurality of images is an image of the inside of the store
  • the location information includes store information related to the store
  • the distribution means is the first from the user terminal. 1 Described in Appendix 1, characterized in that, when a switching instruction is received, an image containing store information different from the store information included in the first position information among the second images is distributed to the user terminal. It is a remote monitoring device of.
  • Appendix 4 In the remote monitoring device according to Appendix 4, when the plurality of images are each provided with attribute information indicating an attribute related to the subject, and the distribution means receives the first switching instruction from the user terminal. , Of the second image, the attribute information including the store information different from the store information included in the first position information and showing the same attribute as the attribute indicated by the attribute information given to the first image.
  • the remote monitoring device according to Appendix 3 characterized in that the given image is distributed to the user terminal.
  • the remote monitoring device includes an acquisition unit that acquires a plurality of time-continuous frame images from the imaging means, and stores the plurality of frame images in the storage means as one of the plurality of images. With at least a part of the one frame image and another frame image including a frame image after the one frame image and the same subject as the subject included in at least a part of the one frame image. With the extraction means for storing at least a part of the other frame image as one of the plurality of images in the storage means, provided that the amount of change in the state of the subject is larger than a predetermined amount.
  • the remote monitoring device according to any one of Supplementary note 1 to 4, wherein the remote monitoring device is provided with.
  • each of the plurality of images is an image of the inside of the store, position information indicating the same position is given to the plurality of images, and different times are provided. At least one of foreign matter, abnormality and missing item is detected by comparing two images to which the time information indicating the above is given, and / or from the image showing the product display shelf among the plurality of images.
  • the remote monitoring device according to any one of Supplementary note 1 to 5, which is provided with a detection means for detecting shelving allocation.
  • Appendix 7 When the remote monitoring device according to Appendix 7 receives the processed first image, which is the processed first image, from the user terminal, the remote monitoring device is applied to another user terminal different from the user terminal.
  • the remote monitoring method described in Appendix 8 includes a storage means for storing a plurality of images to which position information and time information are added, and a distribution means for delivering the first image among the plurality of images to a user terminal.
  • This is a remote monitoring method in a remote monitoring device including, and is given to the first image when a first switching instruction indicating image switching in one direction of the first image is received from the user terminal.
  • Position information indicating a position different from the position indicated by the first position information which is the position information is given, and the same time as the time indicated by the first time information which is the time information given to the first image.
  • the first distribution step of distributing the second image to which the time information indicating the above is added to the user terminal, and the image switching from the user terminal to the other direction intersecting the one direction of the first image are shown.
  • position information indicating the same position as the position indicated by the first position information is given, and time information indicating a time different from the time indicated by the first time information is provided.
  • Appendix 9 The computer program described in Appendix 9 is a computer program that causes a computer to execute the remote monitoring method described in Appendix 8.
  • Appendix 10 The recording medium described in Appendix 10 is a recording medium on which the computer program described in Appendix 9 is recorded.
  • the remote monitoring system includes a remote monitoring device and a user terminal, and the remote monitoring device includes a storage means for storing a plurality of images to which position information and time information are added, and the plurality of storage means.
  • the first image includes a distribution means for distributing the first image to the user terminal, and the distribution means (i) indicates image switching from the user terminal to one direction of the first image.
  • position information indicating a position different from the position indicated by the first position information, which is the position information given to the first image, is given, and is given to the first image.
  • a second image to which the time information indicating the same time as the time indicated by the first time information, which is the time information, is added is distributed to the user terminal, and (ii) the first image of the first image is delivered from the user terminal.
  • the position information indicating the same position as the position indicated by the first position information is given, and the first time information
  • the remote monitoring device has a storage means for storing a plurality of images to which identification information relating to a person as a subject and time information are added, and a first image among the plurality of images.
  • the delivery means includes a delivery means for delivering to the user terminal, and the delivery means (i) receives a first switching instruction indicating image switching in one direction of the first image from the user terminal. Identification information different from the first identification information which is the identification information given to the first image is given, and the same time as the time indicated by the first time information which is the time information given to the first image is set.
  • the second image to which the indicated time information is added is distributed to the user terminal, and (ii) the second switching indicating image switching from the user terminal to the other direction intersecting the one direction of the first image.
  • the user obtains a third image to which the same identification information as the first identification information is given and to which time information indicating a time different from the time indicated by the first time information is given. It is a remote monitoring device characterized by delivering to a terminal.
  • the image display device includes a display means for displaying a first image among a plurality of images to which position information and time information are added, and the above-mentioned.
  • the display means includes an acquisition means for acquiring a switching instruction for switching an image to be displayed, and the display means has (i) the first switching instruction indicating image switching in one direction of the first image.
  • position information indicating a position different from the position indicated by the first position information, which is the position information given to the first image, is given, and is given to the first image.
  • a second image to which time information indicating the same time as the time indicated by the first time information which is time information is added is displayed, and (ii) an image in another direction intersecting with the one direction of the first image.
  • the image display device displays, respectively, a display means for displaying the first image among a plurality of images to which identification information relating to a person as a subject and time information are added, and the display means.
  • the display means includes (i) a first switching instruction indicating image switching in one direction of the first image, which is acquired as the switching instruction.
  • identification information different from the first identification information which is the identification information given to the first image is given, and is indicated by the first time information which is the time information given to the first image.
  • a second image to which time information indicating the same time as the time is added is displayed, and (ii) a first switching instruction indicating image switching in one direction of the first image is acquired as the switching instruction.
  • An image display characterized by displaying a third image to which the same identification information as the first identification information is added and to which time information indicating a time different from the time indicated by the first time information is added. It is a device.
  • the present invention can be appropriately modified within the scope of the claims and within the scope not contrary to the gist or idea of the invention which can be read from the entire specification, and monitoring devices, monitoring methods, computer programs and recording media accompanied by such changes are also made. It is included in the technical idea of the present invention.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Business, Economics & Management (AREA)
  • Library & Information Science (AREA)
  • Data Mining & Analysis (AREA)
  • Databases & Information Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Signal Processing (AREA)
  • Accounting & Taxation (AREA)
  • Finance (AREA)
  • Human Computer Interaction (AREA)
  • Emergency Management (AREA)
  • Development Economics (AREA)
  • Economics (AREA)
  • Marketing (AREA)
  • Strategic Management (AREA)
  • General Business, Economics & Management (AREA)
  • Closed-Circuit Television Systems (AREA)
  • Alarm Systems (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)
PCT/JP2020/014123 2019-06-10 2020-03-27 遠隔監視装置、遠隔監視方法、コンピュータプログラム及び記録媒体 Ceased WO2020250544A1 (ja)

Priority Applications (6)

Application Number Priority Date Filing Date Title
JP2021525922A JP7533458B2 (ja) 2019-06-10 2020-03-27 遠隔監視装置、遠隔監視方法、コンピュータプログラム及び記録媒体
US17/617,040 US20220230441A1 (en) 2019-06-10 2020-03-27 Remote monitoring apparatus, remote monitoring method, computer program and recording medium
US18/237,714 US20230401856A1 (en) 2019-06-10 2023-08-24 Remote monitoring apparatus, remote monitoring method, computer program and recording medium
US18/237,766 US20230401857A1 (en) 2019-06-10 2023-08-24 Remote monitoring apparatus, remote monitoring method, computer program and recording medium
US18/237,778 US20230401858A1 (en) 2019-06-10 2023-08-24 Remote monitoring apparatus, remote monitoring method,computer program and recording medium
US18/659,140 US20240290103A1 (en) 2019-06-10 2024-05-09 Remote monitoring apparatus, remote monitoring method, computer program and recording medium

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2019107762 2019-06-10
JP2019-107762 2019-06-10

Related Child Applications (5)

Application Number Title Priority Date Filing Date
US17/617,040 A-371-Of-International US20220230441A1 (en) 2019-06-10 2020-03-27 Remote monitoring apparatus, remote monitoring method, computer program and recording medium
US18/237,766 Continuation US20230401857A1 (en) 2019-06-10 2023-08-24 Remote monitoring apparatus, remote monitoring method, computer program and recording medium
US18/237,714 Continuation US20230401856A1 (en) 2019-06-10 2023-08-24 Remote monitoring apparatus, remote monitoring method, computer program and recording medium
US18/237,778 Continuation US20230401858A1 (en) 2019-06-10 2023-08-24 Remote monitoring apparatus, remote monitoring method,computer program and recording medium
US18/659,140 Division US20240290103A1 (en) 2019-06-10 2024-05-09 Remote monitoring apparatus, remote monitoring method, computer program and recording medium

Publications (1)

Publication Number Publication Date
WO2020250544A1 true WO2020250544A1 (ja) 2020-12-17

Family

ID=73781781

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2020/014123 Ceased WO2020250544A1 (ja) 2019-06-10 2020-03-27 遠隔監視装置、遠隔監視方法、コンピュータプログラム及び記録媒体

Country Status (3)

Country Link
US (5) US20220230441A1 (https=)
JP (1) JP7533458B2 (https=)
WO (1) WO2020250544A1 (https=)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US12177740B2 (en) * 2021-09-28 2024-12-24 Univrses Ab Managing mobile data gathering agents

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2006139106A (ja) * 2004-11-12 2006-06-01 Noritsu Koki Co Ltd 画像処理システム及び画像処理方法
JP2010119139A (ja) * 2000-10-03 2010-05-27 Olympus Corp 画像再生装置
JP2010252386A (ja) * 2010-06-25 2010-11-04 Olympus Imaging Corp 表示制御装置、カメラ
JP2017084293A (ja) * 2015-10-30 2017-05-18 パナソニックＩｐマネジメント株式会社 施設運営支援装置、ユーザ端末装置、および施設運営支援方法
JP2017162375A (ja) * 2016-03-11 2017-09-14 パナソニックＩｐマネジメント株式会社 商品モニタリング装置、商品モニタリングシステムおよび商品モニタリング方法
JP2017182653A (ja) * 2016-03-31 2017-10-05 パナソニックＩｐマネジメント株式会社 商品モニタリング装置、商品モニタリングシステムおよび商品モニタリング方法

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2014097505A1 (ja) * 2012-12-19 2014-06-26 Ｎｅｃカシオモバイルコミュニケーションズ株式会社 携帯端末、表示制御方法、及びプログラム
US9411831B2 (en) * 2013-03-01 2016-08-09 Facebook, Inc. Photo clustering into moments
KR20150105749A (ko) * 2014-03-10 2015-09-18 삼성전자주식회사 영상을 디스플레이하는 방법 및 장치
US10148918B1 (en) * 2015-04-06 2018-12-04 Position Imaging, Inc. Modular shelving systems for package tracking
US10547776B2 (en) * 2016-09-23 2020-01-28 Apple Inc. Devices, methods, and graphical user interfaces for capturing and recording media in multiple modes
EP3738073B1 (en) * 2018-01-10 2025-06-25 Trax Technology Solutions Pte Ltd. Automatically monitoring retail products based on captured images

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2010119139A (ja) * 2000-10-03 2010-05-27 Olympus Corp 画像再生装置
JP2006139106A (ja) * 2004-11-12 2006-06-01 Noritsu Koki Co Ltd 画像処理システム及び画像処理方法
JP2010252386A (ja) * 2010-06-25 2010-11-04 Olympus Imaging Corp 表示制御装置、カメラ
JP2017084293A (ja) * 2015-10-30 2017-05-18 パナソニックＩｐマネジメント株式会社 施設運営支援装置、ユーザ端末装置、および施設運営支援方法
JP2017162375A (ja) * 2016-03-11 2017-09-14 パナソニックＩｐマネジメント株式会社 商品モニタリング装置、商品モニタリングシステムおよび商品モニタリング方法
JP2017182653A (ja) * 2016-03-31 2017-10-05 パナソニックＩｐマネジメント株式会社 商品モニタリング装置、商品モニタリングシステムおよび商品モニタリング方法

Also Published As

Publication number Publication date
US20230401858A1 (en) 2023-12-14
JPWO2020250544A1 (https=) 2020-12-17
US20220230441A1 (en) 2022-07-21
US20240290103A1 (en) 2024-08-29
US20230401857A1 (en) 2023-12-14
JP7533458B2 (ja) 2024-08-14
US20230401856A1 (en) 2023-12-14

Similar Documents

Publication Publication Date Title
US9049348B1 (en) Video analytics for simulating the motion tracking functionality of a surveillance camera
US20080077510A1 (en) Method And System For Providing Security Surveillance And Shelf Monitoring Functions
AU2017355762B2 (en) Digital signage control device, digital signage control program, and digital signage system
CN111295684A (zh) 货架监视装置、货架监视方法及货架监视程序
US20140003655A1 (en) Method, apparatus and system for providing image data to represent inventory
WO2020077096A1 (en) Systems, method and apparatus for automated inventory interaction
US20170193309A1 (en) Moving information analyzing system and moving information analyzing method
US20140201039A1 (en) System and method for an automated process for visually identifying a product's presence and making the product available for viewing
JP5827642B2 (ja) 監視システム
US20210216951A1 (en) System and Methods for Inventory Tracking
US20240290103A1 (en) Remote monitoring apparatus, remote monitoring method, computer program and recording medium
CN111164531A (zh) 用于监视用无人机的系统、方法和装置
US20140193084A1 (en) Content validation analysis method and apparatus
CN111199410A (zh) 商品管理方法、装置和智能货架
EP3550545A1 (en) Digital signage control apparatus and digital signage control program
US12051217B2 (en) Image processing apparatus, image processing system, and control method
US20140002646A1 (en) Bottom of the basket surveillance system for shopping carts
US11546657B1 (en) Method and system for roles based selection and control of video content
KR20090001680A (ko) 중앙 집중형 광고 시스템 및 그 방법
JP7191358B2 (ja) 表示制御システム及び表示制御装置
US20220343743A1 (en) Display control apparatus, display control method, and program
WO2021090755A1 (ja) 管理装置、管理方法、およびプログラム
US20170286818A1 (en) Interactive item display system
JP5412743B2 (ja) 電子ｐｏｐシステム、サーバ、プログラム
US11017460B1 (en) Electronic shelf system

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20822364

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2021525922

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20822364

Country of ref document: EP

Kind code of ref document: A1