JP6538392B2 - INFORMATION PROCESSING SYSTEM, COMPUTER, INFORMATION PROCESSING DEVICE, INFORMATION PROCESSING PROGRAM, AND INFORMATION PROCESSING METHOD - Google Patents

Description

  The present invention relates to an information processing system, a computer, an information processing apparatus, an information processing program, and an information processing method.

  A video collection device that installs a camera capable of capturing a wide area in a parking lot, analyzes the captured video and detects events such as vehicle entry, parking at a parking location, departure from a parking location, etc. Is disclosed (Patent Document 1).

  There is disclosed a parking lot management apparatus in which cameras are installed at individual parking places in a parking lot and a host computer controls the individual cameras to start and end photographing (Japanese Patent Application Laid-Open No. 2000-112118).

JP 2000-339923 A JP 2001-134898

  However, in the video acquisition device of Patent Document 1, the captured video is divided into a plurality of areas, and an event in each area is detected. Therefore, the software algorithm becomes complicated. In addition, depending on the position and orientation of the vehicle in the video, it is difficult to recognize the car number written on the license plate, so that the event detection accuracy decreases.

  In the parking lot management device of Patent Document 2, cameras to be photographed are sequentially switched. Therefore, it is not possible to detect an event that has occurred at a parking place other than during shooting.

  In one aspect, it is an object of the present invention to provide an information processing system or the like that can accurately record an event using a simple software algorithm.

In the information processing system, the first information processing apparatus and a plurality of second information processing apparatuses respectively disposed in a plurality of vehicle stopping blocks provided in the parking section are connected via a communication network, and the second information The processing device includes a transmitter configured to transmit a still image acquired from a camera built in the car stop block to the first information processing device with the lens directed in the direction in which the parked vehicle enters , the first information processing device A first receiving unit for receiving the still image, and request information transmission for transmitting request information indicating a moving image request to the second information processing apparatus when it is determined that there is a change based on the still image received in time series A second unit, and the second information processing apparatus includes a moving image transmission unit which transmits a moving image of a time zone traced back by a predetermined time period to the first information processing apparatus when the request information is received.

  In one aspect, it is possible to provide an information processing system or the like that can accurately record an event using a simple software algorithm.

It is an explanatory view showing the hardware group of an information processing system. It is an explanatory view showing a hardware group of a client. It is a perspective view which shows the external appearance of a client. It is an explanatory view showing an example of a picture at the time of parking. It is an explanatory view showing an example of a picture with a suspicious person. It is explanatory drawing which shows the example of the image | video in which the vehicle is warehousing next. It is an explanatory view showing an example of a picture with remains. It is an explanatory view showing transition of a situation. It is an explanatory view showing the record layout of record preservation DB It is an explanatory view showing a record layout of still picture DB. It is an explanatory view showing a record layout of empty image DB. It is an explanatory view showing a record layout of parking picture DB. It is a flowchart which shows the flow of a process of an information processing system. It is a flow chart which shows a flow of processing of a subroutine of animation processing. 10 is a flowchart showing a flow of processing of a subroutine of situation determination 1; 10 is a flowchart showing a flow of processing of a subroutine of situation determination 2; 15 is a flowchart showing the flow of processing of the information processing system of the second embodiment. 15 is a flowchart showing a flow of processing of a subroutine of moving image processing of the third embodiment. FIG. 16 is a flowchart showing a flow of processing of a sub-routine for entry / exit determination of the third embodiment. FIG. 18 is an explanatory view showing a hardware group of the information processing system of the fourth embodiment. FIG. 18 is an explanatory view showing a hardware group of a car number recognition device of a fourth embodiment. FIG. 21 is an explanatory drawing showing a record layout of a car number DB of the fourth embodiment. FIG. 18 is an explanatory view showing a record layout of a record storage DB of the fourth embodiment. FIG. 20 is a flowchart showing a flow of processing of a subroutine of moving image processing of the fourth embodiment. FIG. 18 is a flowchart showing a flow of processing of a subroutine of collation of Embodiment 4; FIG. FIG. 18 is an explanatory view showing a hardware group of the information processing system of the fifth embodiment.

First Embodiment
FIG. 1 is an explanatory view showing a hardware group of the information processing system 10. FIG. 2 is an explanatory view showing a hardware group of the client 40. As shown in FIG. FIG. 3 is a perspective view showing the appearance of the client 40. As shown in FIG. The outline of the information processing system 10 according to the present embodiment will be described with reference to FIGS. 1 to 3.

  The information processing system 10 of the present embodiment is a system that records events such as entering and leaving a vehicle to each parking place in the parking lot 12 provided with a plurality of parking places. The information processing system 10 includes a first information processing device 20 and a plurality of second information processing devices 40, 40, 40. In the following description, the first information processing apparatus 20 is described as the server 20, and the second information processing apparatus 40 is described as the client 40.

  The server 20 is connected to a plurality of clients 40, 40, 40... Via the network 11. The network 11 is, for example, a LAN (Local Area Network) in a parking lot connected by a router, a hub or an access point. As the network 11, a public line network, a dedicated line network, the Internet or the like may be used.

  The server 20 includes a central processing unit (CPU) 21, a communication unit 22, a main storage device 23, an auxiliary storage device 24, an input unit, a display unit, a clock unit, and a bus. The server 20 is an information processing apparatus such as a general-purpose personal computer, a tablet, or a large computer installed in a data center.

  The CPU 21 is an arithmetic and control unit that executes a program according to the present embodiment. One or more CPUs or multi-core CPUs are used as the CPU 21. The CPU 21 is connected to hardware units constituting the server 20 via a bus.

  The communication unit 22 communicates with the client 40 via the network 11. The communication unit 22 is used to exchange information with the client 40. The communication unit 22 is, for example, a wireless or wired LAN module.

  The main storage device 23 is a storage device such as a static random access memory (SRAM), a dynamic random access memory (DRAM), a flash memory, or a semiconductor memory disk. The main storage unit 23 temporarily stores information necessary during the process performed by the CPU 21 and a program being executed by the CPU 21.

  The auxiliary storage device 24 is a storage device such as an SRAM, a DRAM, a flash memory, a semiconductor memory disk or a hard disk. The auxiliary storage device 24 stores a program to be executed by the CPU 21, a record storage DB (DataBase) 31, a still image DB 32, a free image DB 34, a moving image DB 33, and a parking image DB 35.

  The input unit is a device such as a mouse, a keyboard, a touch panel, a pen tablet, and a microphone, and is used when the server 20 receives an operation by the user. The display unit is an apparatus such as a liquid crystal display or a head mounted display. The clock unit provides the CPU 21 with the current date and time. It is desirable that the clock unit automatically synchronizes with Japan Standard Time via the network 11.

  The client 40 includes a client CPU 41, a communication unit 42, a main storage device 43, an auxiliary storage device 44, a camera 45, a lighting device 46, and a bus. The client 40 is an information processing apparatus with a photographing function built in a car stop block 49 installed in each parking place.

  The client CPU 41 is an arithmetic and control unit that executes a program according to the present embodiment. For the client CPU 41, one or more CPUs or multi-core CPUs are used. The CPU 21 is connected to hardware units constituting the client 40 via a bus.

  The communication unit 42 communicates with the server 20 via the network 11. The communication unit 42 is used to exchange information with the server 20. The communication unit 42 is, for example, a wireless or wired LAN module.

  The main storage device 43 is a storage device such as an SRAM, a DRAM, a flash memory, or a semiconductor memory disk. The main storage device 43 temporarily stores information required during the process performed by the client CPU 41 and the program being executed by the client CPU 41.

  The camera 45 captures moving and still images. The camera 45 includes a lens 451 and an imaging device 452. The lens 451 is installed facing the direction in which the vehicle enters the parking area. The lens 451 forms an image in the visual field direction on the imaging surface of the imaging element 452. The imaging element 452 photoelectrically converts the formed image, and outputs moving image and still image data of a predetermined format to the bus.

  The illumination device 46 illuminates the surroundings. The illumination device 46 receives and lights an input from a brightness sensor or a clock unit (not shown), illuminates the rear of the vehicle, and secures the brightness necessary for photographing. In addition, we use as lamp of mark.

  The auxiliary storage device 44 is a storage device such as SRAM, DRAM, flash memory, semiconductor memory disk or hard disk. The auxiliary storage device 44 stores a program to be executed by the client CPU 41 and recording data 441.

  At parking places provided in the parking lot 12, car blocking blocks 49 each equipped with a client 40 are installed. Each parking place is given a place ID (IDentification) which is a unique symbol.

  FIG. 4 is an explanatory view showing an example of a video at the time of parking. FIG. 5 is an explanatory view showing an example of an image in which a suspicious person is present. FIG. 6 is an explanatory view showing an example of a video image next to which a vehicle is being received. The image | video which the camera 45 of this Embodiment image | photographs is demonstrated using FIGS. 4-6.

  FIG. 4A shows an image when the vehicle starts to enter a parking location. Since the camera 45 for wide-angle shooting with a horizontal angle of view of about 120 degrees is used, the distance between the dividing lines 56 indicating the boundary with the adjacent parking area widens toward the near side. The rear of the vehicle is visible between the dividing lines 56.

  FIG. 4B shows an image of the vehicle as it enters a parking location. The license plate is clearly photographed. Here, the car number recognition area 51 is an area where the license plate may be included.

  FIG. 4C shows an image when the vehicle is stopped at a predetermined position of the parking place. The underside of the vehicle is visible in most of the field of view, and the image is dark as a whole. The situation of the right and left and the front of the vehicle between the underside of the vehicle and the road is shown. Here, the vehicle peripheral area 52 is an area where the ground around the vehicle or the floor of the parking lot 12 is photographed.

  FIG. 5 shows an image in which a person's foot appears in the immediate vicinity of a parked vehicle. Immediately after the vehicle enters the parking area and immediately before the vehicle leaves the parking area, it can be estimated that it is the foot of a person getting on and off the vehicle. However, if it is time other than that, there is a high possibility that it is a leg of a suspicious person such as, for example, a car raging.

  FIG. 6 shows an image showing the right rear wheel of the vehicle being received at the parking place next to the left. Thereafter, when the vehicles collide with each other, for example, due to an operation error of the driver of the vehicle in storage, an image in which the lower surface of the vehicle shakes is photographed.

FIG. 7 is an explanatory view showing an example of a video with a remains.
FIG. 7 shows an image in which the left baggage is shown in the vacant section where the vehicle is not stopped.

  FIG. 8 is an explanatory view showing the transition of the situation. The top line in FIG. 8 indicates the passage of time. It shows that time goes from the left side of the line to the right side. The character string beginning with T written at the top and bottom of the line indicates the time.

  The status line shows the status of one parking location. The time from the start time of FIG. 8 to the time TA is a vacant state in which the vehicle is not parked at the parking location. The time from the time TA to the time TB is a situation where the vehicle arrives and performs a storage operation to stop at a predetermined position. From time TB to time TC, the vehicle is in a parked state in which the vehicle stops at a predetermined position and does not move. From time TC to time TD, a situation where there is a disturbance such as a suspicious person being photographed as shown in FIG. 5 or a vehicle entering or leaving the next section as being shown in FIG. 6 It is. From time TD to time TE, it is a situation where parking is performed again. From time TE to time TF, there is a situation in which the vehicle has performed an exit operation from the predetermined position. After time TF, the situation is vacant. That is, FIG. 8 shows a series of flows until the vehicle is parked in one vacant parking area for a while and becomes idle again.

  The upward arrow in the line of the still image request indicates that the CPU 21 has requested transmission of a still image to one client 40 via the communication unit 22 and the network 11. The time at which the request was made is indicated by the symbol T0 to T27. In this embodiment, the time from one request to the next is about 10 seconds.

  The client 40 always shoots a moving image while the information processing system 10 is in operation. The client CPU 41 records the video recording data 441 of the moving image for about 25 seconds in the past in the auxiliary storage device 44. The client CPU 41 deletes the old part of the recording data 441 and adds a new image. As a result, the latest video data of 25 seconds is always recorded in the recording data 441.

  When the client CPU 41 receives a request from the server 20, the client CPU 41 captures a still image and transmits it to the server 20 via the communication unit 42 and the network 11. The CPU 21 compares the received still image with the still image acquired one time ago to determine the presence or absence of a change.

  The line of change indicates the determined result. The case of no change is indicated by x, and the case of change is indicated by o. After the still image request at T1, the CPU 21 compares the still image acquired at T0 with the still image acquired at T1. Since the parking place remains empty, the CPU 21 determines that there is no change between the two still images. After the still image request at T2, the CPU 21 similarly determines that there is no change in the still image.

  For example, as shown in FIG. 4A, the still image acquired by the CPU 21 after the request for a still image at T3 is an image of a vehicle in the warehousing operation. The CPU 21 determines that there is a change in the still image acquired at T3 because it is clearly different from the still image at the time of empty vehicle acquired at T2.

  During the loading operation, the position of the vehicle changes as needed, so the CPU 21 determines that there is also a change in the still image acquired after the still image request at T4 and T5. The still image acquired after the still image request at T5 and T6 is, for example, a still image in which the under surface of the vehicle shown in FIG. 4C is enlarged. The CPU 21 determines that there is no change in the still image acquired at T6. The CPU 21 determines that there is no change in the still image acquired after the still image request from T7 to T20.

  As shown in FIG. 5 or 6, for example, a disturbance is photographed in the still image acquired after the request for the still image at T21. The CPU 21 determines that there is a change in the still image acquired at T21, since it is clearly different from the parked still image acquired at T20.

  In the still image acquired after the request for the still image at T22, for example, the lower surface of the vehicle is photographed as shown in FIG. 4C. In this still image, the disturbance captured in the still image acquired at T21 is not reflected. The CPU 21 determines that there is a change in the still image acquired at T22. For example, as shown in FIG. 4C, the lower surface of the vehicle is also photographed in the still image acquired after the request for the still image at T23. The CPU 21 determines that there is no change in the still image acquired at T23.

  For example, as shown in FIG. 4A or FIG. 4B, for example, a vehicle in the process of being taken out is photographed as the still image acquired after the request for the still image at T24. The CPU 21 compares the still image acquired at T24 with the still image acquired at T23, and determines that there is a change.

  A vacant parking place is photographed in the still image acquired after the request for the still image of T25. The CPU 21 compares the still image acquired at T25 with the still image acquired at T24 and determines that there is a change. Thereafter, a vacant parking place is also photographed in the still images acquired after the request for still images at T26 and T27. The CPU 21 determines that there is no change in these still images as compared with the still images acquired one before.

  The upward arrow in the moving image request line indicates that the CPU 21 has requested transmission of moving images from one client 40 via the communication unit 22 and the network 11. The CPU 21 requests transmission of a moving image when it determines that there is a change by comparing two still images as described above. That is, when ○ is described in the line of determination, a request for moving images is made. In addition, since the still image is received and the determination is performed after the request for the still image is made, there is a time difference between the time when the CPU 21 requests the still image and the time when the moving image request is performed.

  When the client CPU 41 receives a request from the server 20, the client CPU 41 transmits, to the server 20, the recording data 441 in which a 25-second moving image is recorded via the communication unit 42 and the network 11. The CPU 21 analyzes the received video recording data 441 to determine the situation. The method of determining the situation will be described later.

  The line of the situation determination indicates the result of the CPU 21 determining the situation. If a video request has not been made, the video is not acquired, and the determination of the situation is not performed. From the analysis result of the moving image acquired after the still image request of T3 and T4 has been made, the CPU 21 determines that it is in a state of being in the warehousing operation for storing the vehicle in the parking place. From the analysis result of the operation acquired after the still image request of T5 is made, the CPU 21 determines that the vehicle is in a parked state where parking is stopped.

  In the moving image acquired after the still image request at T21 and T22 is made, the disturbance generated near the parked vehicle is photographed. From the analysis result of the moving image, the CPU 21 determines that the situation is a caution. From the analysis result of the moving image acquired after the still image request of T24 is made, the CPU 21 determines that the situation is during the delivery. From the analysis result of the moving image acquired after the still image request of T25 is made, the CPU 21 determines that the vehicle is out of the parking place and becomes empty.

  FIG. 9 is an explanatory view showing a record layout of the record storage DB 31. As shown in FIG. The record storage DB 31 is a record storage unit that associates and stores the parking place, the file name of the stored moving image, the time when the moving image was requested, the result of determining the situation, and the description items of the license plate.

  The record storage DB 31 includes a place ID field, a moving image file name field, a time field, a status field, a first recognition result field, and a second recognition result field. The first recognition result field includes a first car number field and a first confidence field. The second recognition result field includes a second car number field and a second confidence field. The record storage DB 31 has one record for one moving image request from the server 20 to the client 40.

  In the place ID field, the place ID given to the parking place where the client 40 which has made the moving image request is installed is recorded. In the moving image file name field, a file name in the moving image DB 33 for storing the moving image acquired by the server 20 in response to the moving image request is recorded. In the time field, the time at which the server 20 requested the client 40 to record a moving image is recorded. In the status field, the result of CPU 21 analyzing the moving image and determining the status is recorded.

  In the first car number field, a first candidate for the car number recognized by the CPU 21 from the image of the license plate in the moving image is recorded. In the first reliability field, the reliability of the car number recorded in the first car number field is recorded. In the second car number field, a second candidate for the car number recognized by the CPU 21 from the image of the license plate in the moving image is recorded. In the second reliability field, the reliability of the car number recorded in the second car number field is recorded. If the license plate can not be detected in the moving image, etc., it is recorded as unrecognizable in the first car number field and the second car number field, and information is recorded in the first reliability field and the second reliability field. I will not.

  Explain car numbers and reliability. The camera 45 is attached to the car stop block 49 in the parking area, so dust and dirt are easily attached. In addition, due to restrictions on installation location and cost, an imaging device 452 having a resolution of several hundred kilo pixels to 2 mega pixels is used. Furthermore, when the license plate of the vehicle which is the subject is dirty, it may be bent. Due to these circumstances, when recognizing a car number from a license plate in a moving image, erroneous recognition may occur. Therefore, the algorithm for recognizing the car number is designed to be able to output a plurality of recognition result candidates and the reliability of each candidate. Description of the car number recognition algorithm will be omitted because various ones have already been disclosed.

  FIG. 10 is an explanatory view showing a record layout of the still image DB 32. As shown in FIG. The still image DB 32 is a DB for recording a still image acquired from the client 40 installed in each parking place. The still picture DB 32 has a place ID field and a still picture field. The still image DB 32 has one record for each parking place in the parking lot 12. In the place ID field, the place ID given to the parking place where the client 40 which has made the moving image request is installed is recorded. In the still picture field, a still picture acquired from the client 40 installed at each parking place is recorded. When the CPU 21 newly acquires a still image, the CPU 21 rewrites the still image recorded in the still image field to the latest still image after performing a predetermined operation described later. As a result, the still image DB 32 always records a still image captured one time ago.

  FIG. 11 is an explanatory view showing a record layout of the free image DB 34. As shown in FIG. The vacant image DB 34 is a DB that records a still image when the vehicle is not parked at each parking location and is vacant. The free image DB 34 has a place ID field and a free image field. The free image DB 34 has one record for each parking place in the parking lot 12. In the place ID field, the place ID given to the parking place where the client 40 which has made the moving image request is installed is recorded. In the vacant image field, a still image photographed by the camera 45 when each parking place is vacant is recorded. In the following description, a still image when the vehicle is not parked will be referred to as a vacant image.

  FIG. 12 is an explanatory view showing a record layout of the parking image DB 35. As shown in FIG. Parking picture DB35 is DB which records a still picture when a vehicle is parked at each parking place. Parking picture DB35 has a place ID field and a parking picture field. The parking image DB 35 has one record for each parking place in the parking lot 12. In the place ID field, the place ID given to the parking place where the client 40 which has made the moving image request is installed is recorded. In the parking image field, a still image captured by the camera 45 while the vehicle is parked at the parking location is recorded.

  The outline of the operation of the present embodiment will be described using FIGS. 8 to 12. The cameras 45 of the plurality of clients 40 connected to the network 11 are respectively shooting moving images. The client CPU 41 stores the latest 25 seconds of video in the video data 441.

  The CPU 21 requests a still image from one client 40 via the communication unit 22 and the network 11. The client CPU 41 receiving the request via the network 11 and the communication unit 42 controls the camera 45 to capture a still image, and transmits the still image to the server 20 via the communication unit 42 and the network 11.

  The CPU 21 searches the still image DB 32 using the parking position at which the client 40 that has requested the still image is installed as a key, and acquires the previous still image. The CPU 21 compares the still image acquired from the still image DB 32 with the still image received from the client 40 to determine the presence or absence of a change.

  When determining the presence or absence of a change, the CPU 21 extracts the contours of two images, for example, and determines the presence or absence of a change in the contour. By doing this, it is possible to extract a change in the state of the parking place, ignoring a minute change such as a change in sunshine in the outdoor parking lot, for example.

  If it is determined that there is no change, the CPU 21 requests the next client 40 for a still image via the communication unit 22 and the network 11. The order in which the CPU 21 requests the client 40 for a still image is predetermined and stored in the auxiliary storage device 24. In the order, for example, according to the arrangement of parking places, the clients installed in adjacent parking places are set as the next clients. In this way, when the human security guard visually monitors the video acquired by the server 20, it is easy to grasp the situation. Also, the physical location of the parking places may be determined independently of the arrangement.

  If it is determined that there is a change, the CPU 21 requests the client 40 that has transmitted the still image to transmit a moving image via the communication unit 22 and the network 11.

  The CPU 21 analyzes the moving image, recognizes the car number described on the number plate taken, and stores the car number of the recognized result and the reliability thereof in the auxiliary storage device 24. In the present embodiment, the CPU 21 stores the first candidate car number and reliability, and the second candidate car number and reliability. When the CPU 21 can not recognize the car number, the CPU 21 stores that effect in the auxiliary storage device 24.

  The CPU 21 analyzes the last part of the moving image acquired from the client 40, for example, the moving image for the last three seconds, and determines whether or not there is a motion. Whether or not there is a motion is determined from the presence or absence of a difference between the images constituting the moving image.

  If it is determined that there is no movement in the last part, the CPU 21 searches the vacant image DB 34 using the parking position at which the client 40 that has requested the moving image is installed as a key, and acquires a vacant image. The CPU 21 searches the parking image DB 35 using the parking position at which the client 40 that has requested the moving image is installed as a key, and acquires a parking image when various vehicles are parked.

  The CPU 21 compares the first part of the moving image, for example, each image forming the first three seconds of moving images with the free image obtained from the free image DB 34 and the parking image obtained from the parking image DB 35, to determine similarity and dissimilarity. . Further, the CPU 21 compares the last part of the moving image, for example, each image constituting the moving image for the last 3 seconds, with the free image obtained from the free image DB 34 and the parking image obtained from the parking image DB 35 to determine similarity or dissimilarity. Do. In addition, about the determination method of the similarity of an image, and non-similarity, since the existing various methods exist, description is abbreviate | omitted.

  If it is determined that the first part of the video is similar to the vacant image and the last part of the video is similar to the parking image, the CPU 21 determines that the vehicle is parked at the parking location. For example, when the vehicle arrives at a parking place and stops at a predetermined place during shooting of a 25-second moving image acquired from the client 40, the CPU 21 determines that parking is in progress.

  If it is determined that the first part of the video is similar to the parked image and the last part of the video is similar to the vacant image, the CPU 21 determines that the vehicle is not parked at the parking location. judge. For example, when a vehicle parked at a parking place starts moving and leaves the parking place while shooting a 25-second moving image acquired from the client 40, the CPU 21 determines that the state is vacant.

  When the first part and the last part of the moving image are a combination other than the above two, the CPU 21 determines that the state is a caution state in which a situation different from a normal state occurs. For example, if a person's foot is visible around a parked vehicle as shown in FIG. 5 or if a vehicle entering or exiting a park next to a parked vehicle is shown as shown in FIG. The CPU 21 determines that the state of caution is required. Whether it is determined that the images in FIG. 5 and FIG. 6 are similar to the parked image or not is determined depending on the similarity determination method of the image. In the present embodiment, either similarity determination method may be used.

  Furthermore, as shown in FIG. 7, even when the left baggage is displayed, the CPU 21 determines that the state of caution is required. However, since the still image does not change when the package continues to be left, the CPU 21 does not request the client 40 for a moving image, and therefore, the status is not determined.

  On the other hand, when it is determined that there is movement in the last part of the moving image, the CPU 21 extracts a vehicle body area in which a vehicle body in each image constituting the moving image is shown using image recognition technology. Description of the image recognition technology is omitted because there are various existing methods.

  The CPU 21 determines whether the area occupied by the vehicle body area is increasing or decreasing from the beginning to the end of the moving image. If it is determined that the vehicle is increasing, the CPU 21 determines that the vehicle is moving to a predetermined stop position and is in the warehousing state. If it is determined that the vehicle is decreasing, the CPU 21 determines that the vehicle is moving out of the predetermined stop position and is in the process of leaving the vehicle. If it can not be determined whether it is increasing or decreasing, the CPU 21 determines that the vehicle is in operation. For example, if the vehicle is moving back and forth while operating the steering wheel without entering a predetermined parking location during shooting of a 25-second moving image acquired from the client 40, it is assumed that the operating state is in progress. The CPU 21 makes a determination.

  The CPU 21 stores the moving image acquired from the client 40 in the moving image DB 33. The moving image DB 33 is a collection of moving image data to which unique file names are given. The CPU 21 records the parking position where the client 40 is installed, the file name of the moving image DB 33, the time when the moving image is requested to the client 40, the condition determined from the moving image, and the car number recognized from the moving picture and the reliability of the car number Record on Thereafter, the CPU 21 requests the next client 40 for a still image via the communication unit 22 and the network 11.

  The CPU 21 repeats the above operation with each client 40 connected via the network 11. After going through all the clients 40, return to the first client 40. The CPU 21 requests the first client 40 for a still image via the communication unit 22 and the network 11. The time required to make a round trip back to the same client 40 changes depending on how many clients 40 request an animation on the way. In the present embodiment, in consideration of the number of clients 40, the frequency of loading and unloading of vehicles, and the communication speed of moving image transmission, it is set to go around in about 10 seconds on average.

  FIG. 13 is a flowchart showing the process flow of the information processing system. The flow of processing performed by the CPU 21 and the client CPU 41 according to the present embodiment will be described with reference to FIG.

  The CPU 21 sets one client 40, which is a communication partner, as the first client 40 (step S501). The CPU 21 acquires an image of a still image captured last time from the still image DB 32 (step S502). At this time, the CPU 21 takes out the record of the still image DB 32 using the parking position where the client 40 of the other party is installed as a key, and acquires the still image recorded in the still image field of the record. When the still image captured last time is not stored in the still image DB 32, blank image data is prepared and used instead of the previously captured still image.

  The CPU 21 requests the client 40 for a still image via the communication unit 22 and the network 11 (step S503). The client CPU 41 receives a request for a still image via the network 11 and the communication unit 22 (step S521).

  The client CPU 41 controls the camera 45 via the bus to capture a still image. The client CPU 41 transmits the photographed still image to the server 20 via the communication unit 42 and the network 11 (step S522).

  The CPU 21 receives the still image transmitted from the client 40 (step S504). The CPU 21 compares the still image acquired from the still image DB 32 in step S502 with the still image received from the client 40, and determines whether there is a change (step S505).

  If it is determined that there is a change (YES in step S505), the CPU 21 starts a subroutine of moving image processing (step S506). The moving image processing subroutine is a subroutine for acquiring the recorded data 441 from the client 40 and analyzing it, and recording the status of the parking place where the client 40 is installed in the record storage DB 31. The process of the moving image process subroutine will be described later.

  After step S506, and when it is determined in step S505 that there is no change (NO in step S505), the CPU 21 stores the still image received in step S504 in the still image DB 32 (step S507). At this time, the CPU 21 replaces the still image recorded in the still image field of the record from which the still image is acquired in step S502 with the still image received in step S504.

  The CPU 21 determines whether the process of the client 40 has been completed (step S508). If it is determined that one cycle has not been made (NO in step S508), the CPU 21 changes the communication partner to the next client 40 (step S509). Thereafter, the CPU 21 returns to step S502.

  If it is determined that one round is made (YES in step S508), the CPU 21 waits until a predetermined time has elapsed by the timer (step S510). After the predetermined time has elapsed, the CPU 21 determines whether to end the process (step S511). The processing is ended, for example, when the business hours of the parking lot 12 are ended or when an instruction for ending is received from the user. If it is determined that the process ends (YES in step S511), the CPU 21 ends the process. If it is determined that the process does not end (NO in step S511), the CPU 21 returns to step S501.

  FIG. 14 is a flowchart showing a flow of processing of a subroutine of moving image processing. The moving image processing subroutine is a subroutine for acquiring the recorded data 441 from the client 40 and analyzing it, and recording the status of the parking place where the client 40 is installed in the record storage DB 31. The flow of processing of the moving image processing subroutine will be described using FIG.

  The CPU 21 requests the client 40 for a moving image via the communication unit 22 and the network 11 (step S531). The client CPU 41 receives the request via the network 11 and the communication unit 42 (step S541). The client CPU 41 transmits the recording data 441 to the server 20 via the communication unit 42 and the network 11 (step S 542).

  The CPU 21 receives the recording data 441 via the network 11 and the communication unit 22 (step S532). The CPU 21 analyzes the car number recognition area 51 in each image of the moving image recorded in the recording data 441 to perform car number recognition for recognizing a symbol written on the license plate (step S533). When performing car number recognition, the CPU 21 auxiliary memory is the most reliable first candidate car number and its reliability among the recognition results and the second most reliable second candidate car number and its reliability. The data is stored in the device 24. The CPU 21 determines whether or not recognition of the car number has succeeded (step S534).

  If it is determined that the car number recognition has succeeded (YES in step S534), the CPU 21 starts a subroutine of situation determination 1 (step S535). The subroutine of the condition determination 1 is a subroutine for analyzing the moving image which has succeeded in the car number recognition to determine the condition of the parking place where the client 40 is installed. If it is determined that the vehicle number recognition has not succeeded (NO in step S534), the CPU 21 starts a subroutine of situation determination 2 (step S536). The subroutine of the condition determination 2 is a subroutine for analyzing the moving image which did not succeed in the car number recognition to determine the condition of the parking place where the client 40 is installed. The processing of each subroutine of situation determination 1 and situation determination 2 will be described later.

  The CPU 21 stores the video recording data 441 received in step S532 in the moving image DB 33 (step S537). The moving picture acquired by the server 20 from each client 40 is sequentially accumulated in the moving picture DB 33.

  The CPU 21 records the information in the record storage DB 31 (step S538). Specifically, the CPU 21 creates a new record in the record storage DB 31 and records information in each field. The contents to be recorded will be described more specifically. The CPU 21 records, in the place ID field, a symbol indicating a parking place where the client 40 that has captured the moving image is installed. The CPU 21 records a file name for identifying the moving image recorded in the moving image DB 33 in the moving image file name field. The CPU 21 records, in the time field, the date and time at which the client 40 requested the moving image. The CPU 21 records the situation determined by the subroutine of the situation determination 1 or the subroutine of the situation determination 2 in the situation field. The CPU 21 records the first candidate car number recognized in step S533 in the first car number field and the reliability in the first reliability field. The CPU 21 records the second candidate car number recognized in step S533 in the second car number field and the reliability in the second reliability field. If the car number can not be recognized in step S533, the CPU 21 records that recognition is not possible in the first car number field and the second car number field, and the first reliability field and the second reliability field are stored. Does not record information. The CPU 21 then ends the process.

  FIG. 15 is a flowchart showing the process flow of the subroutine of situation determination 1. The subroutine of the condition determination 1 is a subroutine for analyzing the moving image which has succeeded in the car number recognition to determine the condition of the parking place where the client 40 is installed. The flow of processing of a subroutine of situation determination 1 will be described using FIG.

  The CPU 21 analyzes the last part of the moving image acquired from the client 40, and determines whether there is a motion (step S551). When it is determined that there is no movement (NO in step S551), the CPU 21 searches the vacant image DB 34 using the parking position at which the client 40 requesting the moving image is installed as a key, and acquires a vacant image (step S552). The CPU 21 determines whether the image of the last still part of the moving image is similar to the empty image (step S553). If it is determined that they are similar (YES in step S553), the CPU 21 determines that the parking place is vacant (step S554), and stores the result in the auxiliary storage device 24. The CPU 21 then ends the process. If it is determined that they are not similar (NO in step S553), the CPU 21 updates the parking image of the location ID of the parking image DB 35 with the image of the last stationary part of the moving image as the parking image (step S 555). The CPU 21 then ends the process.

  If it is determined that the last part of the moving image acquired from the client 40 has a motion (YES in step S551), the CPU 21 determines whether the area occupied by the vehicle body area is increasing (step S561) . If it is determined that the increase is in progress (YES in step S561), the CPU 21 determines that the vehicle is in the warehousing state in which the vehicle is in the parking place (step S562), and the auxiliary storage device 24 Remember. The CPU 21 then ends the process.

  If it is determined that the area occupied by the vehicle body area is not increasing (NO in step S561), the CPU 21 determines whether the area occupied by the vehicle body area is decreasing (step S563). If it is determined that the vehicle is decreasing (YES in step S563), the CPU 21 determines that the vehicle is in the process of leaving the parking area (step S564), and the auxiliary storage device 24 Remember. The CPU 21 then ends the process.

  If it is determined that the area occupied by the vehicle body area is not decreasing (NO in step S563), the CPU 21 determines that the vehicle is in operation (step S565), and stores it in the auxiliary storage device 24. The CPU 21 then ends the process.

  FIG. 16 is a flowchart showing the process flow of the subroutine of situation determination 2. The subroutine of the condition determination 2 is a subroutine for analyzing the moving image which did not succeed in the car number recognition to determine the condition of the parking place where the client 40 is installed. The flow of processing of the subroutine of situation determination 2 will be described using FIG.

  The CPU 21 searches the vacant image DB 34 using the parking position at which the client 40 which has requested the moving image is installed as a key, and acquires the vacant image (step S571). The CPU 21 searches the parking image DB 35 using the parking position at which the client 40 that has requested the moving image is installed as a key, and acquires a parking image when the vehicle is parked (step S572).

  The CPU 21 compares the image of the last still part of the moving image with the empty image to determine whether they are similar (step S573). If it is determined that they are similar (YES in step S573), the CPU 21 determines that the parking place is vacant (step S574), and stores it in the auxiliary storage device 24. The CPU 21 then ends the process.

  When it is determined that the image of the last stationary part of the moving image is not similar to the vacant image (NO in step S573), the CPU 21 determines that the image of the last stationary part of the moving image is a parked image. It compares and it is determined whether it is similar (step S575).

  If it is determined that the image is similar to the parking image (YES in step S575), the CPU 21 determines that the vehicle is parked in the parking place (step S576), and the auxiliary storage device 24 Remember to The CPU 21 then ends the process. If it is determined that the image is not similar to the parking image (NO in step S575), the CPU 21 determines that the situation requires caution (step S577), and stores it in the auxiliary storage device 24. The CPU 21 then ends the process.

  According to the present embodiment, there is provided an information processing system and the like which accurately detects and records the occurrence of an event such as the movement of a vehicle or the operation of a person or another vehicle around the vehicle using a simple software algorithm. be able to.

  When an incident such as theft or hit-and-run within the parking lot 12 occurs, the moving image at the time of the incident, the vehicle parked around the incident occurrence site by referring to the record storage DB 31 of the present embodiment Etc. information can be easily extracted.

  When there is a driver who has forgotten the place where the vehicle was parked, the parked place can be extracted from the car number by referring to the record storage DB 31 of the present embodiment. At this time, it is possible to make it easy for the driver to find the parked place by lighting the lighting device 46 at the parked place.

  When there is a notification of a lost item in the parking lot 12, it is possible to extract a possible location of the lost item by referring to the record storage DB 31 of the present embodiment. Also, by referring to the moving image when a lost item occurs, it is possible to confirm whether the person who has notified is the owner.

  The records in the record storage DB 31 may be deleted by the CPU 21 after a certain period, for example, one week has passed. By doing this, the capacity of the auxiliary storage device 24 can be reduced.

  If there is a wall behind each parking location, the client 40 may be attached to the wall. By doing so, not only the feet of the person but also the body, face, personal belongings, etc. can be photographed in the image of FIG.

  When using the imaging device 452 capable of photographing in a dark place, the illumination device 46 may not be provided when sufficient illumination is installed in the parking lot 12 and when it is not used at night.

  The client 40 may include two cameras, a camera 45 for moving image shooting and a camera 45 for still image shooting. In this case, it is desirable that the fields of view of the moving image shooting camera 45 and the still image shooting camera 45 be substantially the same.

  The client 40 may not have the camera 45. In this case, the client 40 controls the external camera 45 installed in the parking area by communication using a wireless LAN, a wired LAN, wireless HDMI (registered trademark) or the like, and acquires still images and moving images. When the camera 45 is not provided, a general-purpose personal computer, a tablet or the like can be used as the client 40.

  About 10 seconds of the still image shooting time interval and 25 seconds of the moving image storage time to the recording data 441 are examples, and any other time can be set. At this time, it is desirable that the storage time of the recorded data 441 be longer than the maximum value of the still image shooting interval.

  The client 40 may extract a moving image after the time when the still image was transmitted to the server 20 two times before at step S 542 and transmit it to the server. In this way, the amount of moving images to be transmitted can be reduced, so the load on the network 11 and the CPU 21 can be reduced. Also, the capacity of the moving picture DB 33 can be saved.

  The respective subroutines of step S534, step S535 and step S536 may be omitted, and the status field of the record storage DB 31 may be omitted. In this case, since the moving image and the car number when an event occurs are recorded, the user may view the moving image as needed. By doing this, the load on the CPU 21 can be reduced.

  A timer may be inserted between step S508 and step S509 to keep the still image shooting interval constant. In this case, the moving image storage time may be the same as the still image shooting interval.

  Alternatively, the timer in step S510 may be omitted, and as soon as the client 40 has made a round, the first client 40 may return to request a still image.

  When the moving image is recorded by the same client 40 in the previous time, when the recorded data 441 is stored in step S537, a portion overlapping with the previous moving image may be removed. By doing so, the storage capacity of the auxiliary storage device 24 can be saved. In addition, when an incident occurs and the user browses the video, it is not necessary to view the same video redundantly.

  In the subroutine of situation determination 2, after determining that the situation requires attention, a more detailed situation may be determined using image recognition technology. For example, when the image of FIG. 6 is captured, it may be determined that the vehicle is parked in the section next to the left, and may be recorded in the status field of the record storage DB 31. In this way, when an incident or the like occurs and the record storage DB 31 is checked, necessary information can be easily extracted.

Second Embodiment
The present embodiment relates to an information processing system in which the client 40 determines the presence or absence of movement within the field of view of the camera 45. The description of the parts common to the first embodiment will be omitted. The hardware configuration of the present embodiment is the same as that of the first embodiment, but the still image DB 32 is not used.

  FIG. 17 is a flowchart showing the process flow of the information processing system of the second embodiment. The flow of processing according to this embodiment will be described using FIG.

  The client CPU 41 sets the movement flag to OFF, which is the initial value (step S601). The client CPU 41 monitors a moving image captured by the camera 45, and determines whether there is a motion in the captured field of view (step S602). If it is determined that there is a motion (YES in step S602), the client CPU 41 sets the motion flag to ON (step S603). If it is determined that there is no movement (NO in step S602) and after step S603, the client CPU 41 returns to step S602.

  The CPU 21 sets one client 40, which is a communication partner, as the first client 40 (step S611). The CPU 21 requests the client 40 to transmit the motion flag via the communication unit 22 and the network 11 (step S612).

  The client CPU 41 receives a request from the server 20 via the network 11 and the communication unit 42 (step S604). The client CPU 41 transmits the motion flag to the server 20 via the communication unit 42 and the network 11 (step S605).

  The CPU 21 receives the motion flag via the network 11 and the communication unit 22 (step S613). The CPU 21 determines whether the received motion flag is ON (step S614). If it is determined that the movement flag is ON (YES in step S614), the CPU 21 starts a subroutine for moving image processing (step S615). The moving image processing subroutine uses the same subroutine as that of FIG. 14 of the first embodiment.

  The CPU 21 determines whether the process of the client 40 has been completed (step S616). If it is determined that one round trip has not been made (NO in step S616), the CPU 21 changes the communication partner to the next client 40 (step S619). Thereafter, the CPU 21 returns to step S612.

  If it is determined that one round is made (YES in step S616), the CPU 21 waits until a predetermined time has elapsed by the timer (step S617). After the predetermined time has elapsed, the CPU 21 determines whether to end the process (step S618). If it is determined that the process ends (YES in step S618), the CPU 21 ends the process. If it is determined that the process does not end (NO in step S618), the CPU 21 returns to step S611.

  In the present embodiment, since the still image is not transferred between the server 20 and the client 40, the load on the network 11 can be reduced. Further, since the process of determining the transfer time of the still image and the presence or absence of the change of the still image is unnecessary, the load on the server 20 can be reduced. Furthermore, since the still image DB 32 is unnecessary, the capacity of the auxiliary storage device 24 can be saved.

Third Embodiment
The present embodiment relates to an information processing system that determines whether a vehicle is in or out of storage using a license plate detected when performing car number recognition. The description of the parts common to the first embodiment will be omitted.

  FIG. 18 is a flowchart showing a flow of processing of a subroutine of moving image processing of the third embodiment. The subroutine shown in FIG. 18 is a subroutine used in place of the subroutine shown in FIG. A flow of processing of a subroutine of moving image processing of the present embodiment will be described using FIG.

  The CPU 21 requests the client 40 for a moving image via the communication unit 22 and the network 11 (step S531). The client CPU 41 receives the request via the network 11 and the communication unit 42 (step S541). The client CPU 41 transmits the recording data 441 to the server 20 via the communication unit 42 and the network 11 (step S 542).

  The CPU 21 receives the recording data 441 via the network 11 and the communication unit 22 (step S532). The CPU 21 analyzes the car number recognition area 51 in each image of the moving image recorded in the recording data 441 to recognize the car number which is a symbol described on the number plate (step S533), and the auxiliary storage device 24 Remember.

  The CPU 21 determines whether the recognition of the car number has succeeded (step S534). If it is determined that the vehicle number has been recognized successfully (YES in step S534), the CPU 21 starts a subroutine of storage determination (step S632). The sub-routine for entering / removing determination is a sub-routine for determining from the image of the number plate whether the vehicle is in storage or out. A flow of processing of a subroutine of entry / exit determination will be described later.

  If it is determined that the car number has not been successfully recognized (NO in step S534), the CPU 21 determines that the situation requires caution (step S633), and stores it in the auxiliary storage device 24. After the end of step S632 and after the end of step S633, the CPU 21 stores the video recording data 441 received in step S532 in the moving image DB 33 (step S537).

  The CPU 21 records the information in the record storage DB 31 (step S538). The CPU 21 then ends the process.

  FIG. 19 is a flow chart showing a flow of processing of a subroutine of entry / exit determination of the third embodiment. The sub-routine for entering / removing determination is a sub-routine for determining from the image of the number plate whether the vehicle is in storage or out. The flow of the processing of the storage / storage determination subroutine will be described using FIG.

  The CPU 21 acquires images before and after the image whose car number has been successfully recognized in step S533, for example, moving images of 0.5 seconds before and after each image whose car number has been recognized, from the recording data 441 received in step S532 (step S641) ). The CPU 21 extracts the outer frame of the license plate from the acquired image (step S642). The CPU 21 determines whether the outer frame of the number plate has become larger as time passes (step S643). If it is determined that the size of the vehicle is large (YES in step S643), the CPU 21 determines that the vehicle is moving to a predetermined stop position and is in the warehousing state (step S647). . The CPU 21 then ends the process.

  If it is determined that the outer frame of the number plate does not increase with the elapse of time (NO in step S643), the CPU 21 determines whether the outer frame of the number plate decreases with the elapse of time ( Step S644). If it is determined that the size of the vehicle has become smaller (YES in step S644), the CPU 21 determines that the vehicle is moving from the predetermined stop position and is in the process of being taken out (step S645). The CPU 21 then ends the process.

  If it is determined that the outer frame of the number plate has not become smaller with the passage of time (NO in step S644), the CPU 21 can not determine whether the vehicle is in storage or out. Therefore, CPU 21 determines that the situation is unknown (in step S646). The CPU 21 then ends the process.

  According to the present embodiment, it is possible to determine the warehousing and the warehousing by using the license plate extraction technology which has been widely used when the vehicle number recognition is already performed and the proven license plate is used, so highly reliable determination can be performed. .

Fourth Embodiment
The present embodiment relates to an information processing system that enhances the accuracy of recording by combining a car number recognition result using a high-performance camera installed at the entrance of the parking lot 12 and the client 40 installed at each parking location. The description of the parts common to the first embodiment will be omitted.

  FIG. 20 is an explanatory view of a hardware group of the information processing system according to the fourth embodiment. FIG. 21 is an explanatory view showing a hardware group of the car number recognition device 60 of the fourth embodiment. The hardware configuration of the information processing system 10 of the present embodiment will be described using FIGS. 20 and 21. FIG.

  The information processing system 10 according to the present embodiment includes a server 20, a network 11, a plurality of clients 40 connected to the server 20 via the network 11, and a car number recognition device 60 connected to the server 20 via the network 11. Equipped with

  The auxiliary storage device 24 is a storage device such as an SRAM, a DRAM, a flash memory, a semiconductor memory disk or a hard disk. The auxiliary storage device 24 stores a program to be executed by the CPU 21, a record storage DB 31, a still image DB 32, a free image DB 34, a moving image DB 33, a parking image DB 35 and a car number DB 36.

  The car number recognition device 60 includes a gate CPU 61, a communication unit 62, a main storage device 63, an auxiliary storage device 64, a gate camera 65, and a bus. The car number recognition device 60 is installed near the gate at the entrance of the parking lot 12 and is a device that photographs the number plate of the vehicle entering and exiting, recognizes the described car number with high accuracy, and outputs it.

  The gate CPU 61 is an arithmetic and control unit that controls the car number recognition device 60 and performs car number recognition. For the gate CPU 61, one or more CPUs or multi-core CPUs are used. The gate CPU 61 is connected to the hardware units constituting the car number recognition device 60 via a bus.

  The communication unit 62 communicates with the server 20 via the network 11. The communication unit 62 is used to exchange information with the server 20. The communication unit 62 is, for example, a wireless or wired LAN module.

  The main storage device 63 is a storage device such as an SRAM, a DRAM, a flash memory, or a semiconductor memory disk. The main storage device 63 temporarily stores necessary information in the middle of the processing performed by the gate CPU 61 and the program being executed by the gate CPU 61.

  The auxiliary storage device 64 is a storage device such as an SRAM, a DRAM, a flash memory, a semiconductor memory disk or a hard disk. The auxiliary storage device 64 stores a program to be executed by the gate CPU 61.

  The gate camera 65 shoots a moving image. The gate camera 65 is a camera installed at a place where the license plate of the vehicle passing through the entrance of the parking lot 12 can be clearly photographed. The gate camera 65 is a high-quality camera using a high resolution lens and an imaging element with a large number of pixels, and also captures and records a driver and a passenger.

  Since the image obtained by clearly capturing the license plate using the high-quality gate camera 65 is used, the accuracy of vehicle number recognition of the vehicle number recognition device 60 is determined using the vehicle number using the image captured by the camera 45 of the client 40 Much higher than the recognition accuracy. The car number recognition accuracy of the car number recognition device 60 of the present embodiment is about 99%. Vehicles in which the vehicle number recognition device 60 of the present embodiment can not recognize the vehicle number are, for example, vehicles in which the number plate is very dirty with dirt or dust, vehicles in which the number plate is bent, or the like. In addition, since the existing various algorithm is disclosed as the method of the vehicle number recognition which gate CPU61 performs, description is abbreviate | omitted. When the car number of its own vehicle is forgotten, it is possible to recognize the car number by comparing it with the recorded image of the gate camera 65.

  Further, the gate CPU 61 determines whether the car number recognized from the movement of the photographed image is that of the entering vehicle or that of the participating vehicle. The vehicle number recognized by the gate CPU 61 and the distinction between entry and exit are notified to the server 20 via the communication unit 62 and the network 11.

  FIG. 22 is an explanatory drawing showing the record layout of the car number DB 36 of the fourth embodiment. The car number DB 36 is a DB that records the car number recognized by the car number recognition device 60 and the reliability. The car number DB 36 has a car number field and a reliability field. The car number DB 36 has one record for each vehicle entering the parking lot 12. In the car number field, the car number recognized by the gate CPU 61 is recorded. In the reliability field, the reliability of recognition of the car number recorded in the car number field is recorded.

  When the CPU 21 receives the car number of the vehicle to be entered from the car number recognition device 60, the CPU 21 creates a new record in the car number DB 36. The CPU 21 records the received car number in the car number field of the new record. When the CPU 21 receives the car number of the participating vehicle from the car number recognition device 60, the CPU 21 searches the car number DB 36 using the received car number as a key, and deletes the extracted record. From the above, the car number of the vehicle entering the site of the parking lot 12 is recorded in the car number DB 36.

  FIG. 23 is an explanatory drawing showing the record layout of the record storage DB 31 of the fourth embodiment. The record storage DB 31 is a record storage unit that associates and stores the parking place, the file name of the stored moving image, the time when the moving image was requested, the result of determining the situation, and the description items of the license plate.

  The record storage DB 31 includes a place ID field, a moving image file name field, a time field, a status field, a car number field, a matching result field, and a reliability degree field. The record storage DB 31 has one record for one moving image request from the server 20 to the client 40.

  In the place ID field, the place ID given to the parking place where the client 40 which has made the moving image request is installed is recorded. In the moving image file name field, a file name in the moving image DB 33 for storing the moving image acquired by the server 20 in response to the moving image request is recorded. In the time field, the time at which the server 20 requested the client 40 to record a moving image is recorded. In the status field, the result of CPU 21 analyzing the moving image and determining the status is recorded.

  The car number determined by the CPU 21 is recorded in the car number field. If the CPU 21 can not recognize the car number from the image received from the client 40, the car number field is recorded as unrecognizable.

  In the collation result field, the collation result when the CPU 21 collates whether the car number described in the car number field is recorded in the car number DB 36 is recorded. When the CPU 21 can not recognize the car number from the image received from the client 40, the collation is not performed, and the information is not recorded in the collation result field.

  In the reliability field, the reliability of the car number recorded in the car number field is recorded.

  FIG. 24 is a flowchart showing a flow of processing of a subroutine of moving image processing of the fourth embodiment. The subroutine shown in FIG. 24 is a subroutine used in place of the subroutine shown in FIG. 14 or the subroutine shown in FIG. A flow of processing of a subroutine of moving image processing of the present embodiment will be described using FIG.

  The CPU 21 requests the client 40 for a moving image via the communication unit 22 and the network 11 (step S531). The client CPU 41 receives the request via the network 11 and the communication unit 42 (step S541). The client CPU 41 transmits the recording data 441 to the server 20 via the communication unit 42 and the network 11 (step S 542).

  The CPU 21 receives the recording data 441 via the network 11 and the communication unit 22 (step S532). The CPU 21 analyzes the car number recognition area 51 in each image of the moving image recorded in the recording data 441 to recognize the car number which is a symbol described on the number plate (step S533), and the auxiliary storage device 24 Remember. At this time, the CPU 21 stores in the auxiliary storage device 24 the most reliable first candidate car number and its reliability among the recognition results and the second most reliable second candidate car number and its reliability. Do.

  The CPU 21 determines whether the recognition of the car number has succeeded (step S534). If it is determined that the car number has been recognized successfully (YES in step S534), the CPU 21 starts a subroutine for comparison (step S661). The collation subroutine is a subroutine for collating whether the car number recognized in the moving image acquired from the client 40 is recorded in the car number DB 36 or not. The flow of the process of the collation subroutine will be described later. The CPU 21 starts a subroutine of situation determination 1 (step S535). The subroutine of situation determination 1 uses the same subroutine as the subroutine described using FIG.

  If it is determined that the car number has not been successfully recognized (NO in step S534), the CPU 21 starts a subroutine of situation determination 2 (step S536). The subroutine of situation determination 2 uses the same subroutine as the subroutine described using FIG.

  The CPU 21 stores the video recording data 441 received in step S532 in the moving image DB 33 (step S537).

  The CPU 21 records the information in the record storage DB 31 (step S662). Specifically, the CPU 21 creates a new record in the record storage DB 31 and records information in each field. The contents to be recorded will be described more specifically. The CPU 21 records, in the place ID field, a symbol indicating a parking place where the client 40 that has captured the moving image is installed. The CPU 21 records a file name for identifying the moving image recorded in the moving image DB 33 in the moving image file name field. The CPU 21 records, in the time field, the date and time at which the client 40 requested the moving image. The CPU 21 records the situation determined by the subroutine of the situation determination 1 or the subroutine of the situation determination 2 in the situation field. The CPU 21 records the car number collated in the car number field in the collation subroutine. If the recognition of the car number is not successful in step S533, the CPU 21 records that recognition is not possible in the car number field.

  The CPU 21 records the collation result collated in the collation subroutine in the collation result field. The CPU 21 records the reliability of the car number described in the car number field in the reliability field. If the recognition of the car number is not successful in step S533 (NO in step S534) and the verification subroutine is not activated, the CPU 21 does not record information in the reliability field. The CPU 21 then ends the process.

  FIG. 25 is a flowchart showing the flow of the process of the subroutine for collation of the fourth embodiment. The collation subroutine is a subroutine for collating whether the car number recognized in the moving image acquired from the client 40 is recorded in the car number DB 36 or not. The flow of processing of the subroutine of collation will be described using FIG.

  The CPU 21 determines whether the car number corresponding to the first candidate car number recognized in step S533 is in the car number field of the car number DB 36 (step S671). If it is determined that there is a car number that matches the first candidate car number (YES in step S671), the CPU 21 determines that the car number of the vehicle captured in the moving image acquired from the client 40 is the first candidate car number It is determined that there is (step S 672) and stored in the auxiliary storage device 24. The CPU 21 determines that the collation result is "yes" (step S673). The CPU 21 acquires the reliability recorded in the reliability field of the record in which the first candidate car number is recorded. The CPU 21 stores the car number, the reliability, and the determination result in the auxiliary storage device 24. The CPU 21 ends the process.

  If it is determined that there is no vehicle number that matches the first candidate vehicle number (NO in step S671), the CPU 21 checks the vehicle number of the second candidate vehicle number recognized in step S533 that matches the vehicle number DB36 It is determined whether it is in the field (step S676). If it is determined that there is a car number that matches the second candidate car number (YES in step S 676), the CPU 21 determines that the car number of the vehicle photographed in the moving image acquired from the client 40 is the second candidate car number It is determined that there is (step S677) and stored in the auxiliary storage device 24. The CPU 21 determines that the collation result is "yes" (step S678). The CPU 21 acquires the reliability recorded in the reliability field of the record in which the second candidate car number is recorded. The CPU 21 stores the car number, the reliability, and the determination result in the auxiliary storage device 24. The CPU 21 ends the process.

  If it is determined that there is no vehicle number that matches the second candidate vehicle number (NO in step S 676), the CPU 21 selects a vehicle number similar to the first candidate or second candidate vehicle number recognized in step S 533. It is determined whether it is in the car number field of DB 36 (step S 681). The similar car number means, for example, a car number different by one letter from the car number of the first candidate or the second candidate, a car number including characters having a similar shape, and the like.

  If it is determined that there is a similar car number (YES in step S 681), the CPU 21 controls the car number of the vehicle captured in the moving image acquired from the client 40 to be a similar car number recorded in the car number DB 36 (Step S682), and stored in the auxiliary storage device 24. The CPU 21 determines that the collation result is "yes" (step S683). The CPU 21 acquires the reliability recorded in the reliability field of the record in which the similar car number is recorded. The CPU 21 stores the car number, the reliability, and the determination result in the auxiliary storage device 24. The CPU 21 ends the process.

  When it is determined that there is no similar car number (NO in step S681), the CPU 21 determines that the car number of the vehicle captured in the moving image acquired from the client 40 is the first candidate car number (step S686), store in the auxiliary storage device 24. The CPU 21 determines that the collation result is "absent" (step S687). The CPU 21 stores the car number, the reliability obtained in step S533, and the determination result in the auxiliary storage device 24. The CPU 21 ends the process.

  According to the present embodiment, since the car number recognized by the car number recognition device 60 having high car number recognition accuracy is used, highly reliable information can be recorded in the record storage DB 31.

  When using this information processing system in the parking lot 12 where vehicles to be used beforehand are defined, such as monthly parking lot, employee private parking lot, the car number of the vehicle using the parking lot 12 in the car number DB 36 You may record it. In this case, the reliability field of the car number DB 36 is unnecessary, and the reliability acquired in step S533 is recorded in the reliability field of the record storage DB 31. By doing this, it is not necessary to record and delete the record in the car number DB 36, so the load on the CPU 21 can be reduced. When a vehicle not recorded in the car number DB 36 is parked, the parking lot manager may be notified as unauthorized parking. Since the presence of the unauthorized parking vehicle and the parking position can be reported, the parking lot manager can easily confirm the unauthorized parking vehicle and take necessary measures.

Fifth Embodiment
The fifth embodiment relates to a mode in which the information processing system 10 is realized by operating the server 20 using a general-purpose computer and the information processing program 71 in combination. FIG. 26 is an explanatory view of a hardware group of the information processing system 10 according to the fifth embodiment. The configuration of the present embodiment will be described using FIG. The description of the parts common to the first embodiment will be omitted.

  An information processing system 10 according to the present embodiment includes a server 20, a network 11, and a plurality of clients 40 connected to the server 20 via the network 11.

  The server 20 includes a CPU 21, a communication unit 22, a main storage device 23, an auxiliary storage device 24, a reading unit 25, an input unit, a display unit, a clock unit, and a bus. The server 20 is an information processing apparatus such as a general-purpose personal computer, a tablet, or a large computer installed in a data center.

  The information processing program 71 is recorded on a portable recording medium 72. The CPU 21 reads the information processing program 71 via the reading unit 25 and stores the information processing program 71 in the auxiliary storage device 24. The CPU 21 may also read the information processing program 71 stored in the semiconductor memory 73 such as a flash memory mounted in the server 20. Furthermore, the CPU 21 may download the information processing program 71 from another server computer (not shown) connected via the communication unit 22 and the network 11 and store the information processing program 71 in the auxiliary storage device 24.

  The CPU 21 reads the information processing program 71 for executing the various software processes described above from the portable recording medium 72 or the semiconductor memory 73 or downloads it from another server computer via the network 11. The CPU 21 stores necessary modules in the information processing program 71 in the auxiliary storage device 44 of the client 40 via the communication unit 22 and the network 11.

  The information processing program 71 is installed as a control program of the server 20 using the client 40 and a general-purpose computer, and is loaded and executed on the main storage device 43 and the main storage device 23. Thus, the server 20 and the client 40 function as the information processing system 10 described above.

  Note that a program that operates emphasizing the information processing program 71 may be stored in the auxiliary storage device 44 of the client 40 in advance. In this case, the CPU 21 does not store the module in the information processing program 71 in the auxiliary storage device 44 of the client 40.

  Further, the following appendices will be disclosed regarding the above embodiment.

(Supplementary Note 1)
In an information processing system in which a first information processing device and a plurality of second information processing devices are connected via a communication network,
The second information processing apparatus is
A transmission unit configured to transmit a still image acquired from a camera to the first information processing apparatus;
The first information processing apparatus is
A first receiving unit that receives the still image;
A request information transmission unit that transmits request information indicating a request for a moving image to the second information processing apparatus when it is determined that there is a change based on a still image received in time series;
The second information processing apparatus is
An information processing system, comprising: a moving image transmitting unit that transmits a moving image of a time zone traced back by a predetermined time to the first information processing apparatus when the request information is received.

(Supplementary Note 2)
In an information processing system in which a first information processing device and a plurality of second information processing devices are connected via a communication network,
The second information processing apparatus is
A detection recording unit that detects and records the presence or absence of a change in the image acquired from the camera;
The first information processing apparatus is
A first receiving unit that receives the information recorded in the detection recording unit;
And a request information transmission unit that transmits request information indicating a request for a moving image to the second information processing apparatus when it is determined that the information has a change.
The second information processing apparatus is
An information processing system, comprising: a moving image transmitting unit that transmits a moving image of a time zone traced back by a predetermined time to the first information processing apparatus when the request information is received.

(Supplementary Note 3)
The second information processing apparatus includes a storage unit that stores a moving image up to a predetermined time after the shooting time acquired from the camera.
The information processing system according to Appendix 1 or 2, wherein the moving image transmission unit transmits the moving image stored in the storage unit.

(Supplementary Note 4)
The information processing system according to any one of Appendixes 1 to 3, wherein the first information processing apparatus includes a moving image storage unit that stores the received moving image.

(Supplementary Note 5)
The information processing system according to any one of appendices 1 to 4, wherein the first information processing apparatus includes a switching unit that sequentially switches the second information processing apparatus of the communication partner in a predetermined order.

(Supplementary Note 6)
The first information processing apparatus includes a recording and storing unit that associates and stores the moving image transmitted from the moving image transmitting unit, the symbol extracted from the moving image, and the second information processing apparatus that has captured the moving image. The information processing system according to any one of 5.

(Appendix 7)
The first information processing apparatus includes: a group recording unit that records a group of symbols;
And a collating unit that collates whether or not the extracted symbol is included in the group.
The information processing system according to claim 6, wherein the recording storage unit records the collation result of the collation unit in association with the moving image.

(Supplementary Note 8)
The camera is fixed at the parking place of the parking lot.
The information processing system according to Appendix 6 or 7, wherein the symbol is a car number displayed on a license plate of a vehicle parked at a parking location.

(Appendix 9)
The first information processing apparatus includes a car number recording unit that records a car number.
And a verification unit that verifies whether the extracted symbol is recorded in the car number recording unit.
The information processing system according to any one of Appendixes 6 to 8, wherein the recording storage unit records the collation result of the collation unit in association with the moving image.

(Supplementary Note 10)
The information processing system according to appendix 9, wherein the car number recording unit records the car number of the vehicle existing in the parking lot.

(Supplementary Note 11)
The first information processing apparatus includes a condition determination unit that determines a parking condition of a parking place where the moving image is captured from the moving image,
The information processing system according to any one of appendices 6 to 10, wherein the recording storage unit records the determination result of the situation determination unit in association with the moving image.

(Supplementary Note 12)
In a computer connected to a plurality of information processing apparatuses via a communication network,
A receiver configured to receive a still image acquired by the information processing apparatus from a camera;
A request information transmission unit that transmits request information indicating a request for a moving image to the information processing apparatus when it is determined that there is a change based on a still image received in time series;
A computer, comprising: a moving image receiving unit that receives a moving image when a moving image in a time zone dated back by a predetermined time is transmitted by the information processing apparatus that has received the request information.

(Supplementary Note 13)
A video acquisition unit that acquires a video from a camera;
A storage unit that stores the moving image up to a predetermined time after the shooting time;
An information processing unit including information indicating whether or not there is a change in a still image shot by the camera or a moving image shot by the camera, or a video information transmitting unit transmitting the moving image stored in the storage unit in response to the received signal; apparatus.

(Supplementary Note 14)
Acquire a still image from the information processing apparatus,
Determining the presence or absence of a change based on the still image acquired in time series;
When it is determined that there is a change, request information indicating a request for a moving image is transmitted to the information processing apparatus,
An information processing program that causes a computer to execute a process of receiving a moving image when a moving image in a time zone dated back by a predetermined time is transmitted by the information processing apparatus that has received the request information.

(Supplementary Note 15)
Acquire a still image from the information processing apparatus,
Determining the presence or absence of a change based on the still image acquired in time series;
When it is determined that there is a change, request information indicating a request for a moving image is transmitted to the information processing apparatus,
An information processing method for receiving the moving image when a moving image in a time zone dated back by a predetermined time is transmitted by the information processing apparatus having received the request information;

The technical features (component requirements) described in the respective embodiments can be combined with each other, and by combining, new technical features can be formed.
It should be understood that the embodiments disclosed herein are illustrative in all respects and not restrictive. The scope of the present invention is indicated not by the meaning described above but by the claims, and is intended to include all modifications within the meaning and scope equivalent to the claims.

10 information processing system 11 network 12 parking lot 20 server 21 CPU
22 communication unit 23 main storage device 24 auxiliary storage device 25 reading unit 31 record storage DB
32 still picture DB
33 Movie DB
34 Free Image DB
35 Parking image DB
36 car number DB
40 client 41 client CPU
42 communication unit 43 main storage unit 44 auxiliary storage unit 441 recording data 45 camera 49 car stop block 60 car number recognition unit 61 gate CPU
62 Communication Unit 63 Main Storage Device 64 Auxiliary Storage Device 65 Gate Camera 71 Information Processing Program 72 Portable Recording Medium

Claims (8)

In an information processing system in which a first information processing device and a second information processing device respectively disposed in a plurality of vehicle stop blocks provided in a parking section are connected via a communication network,
The second information processing apparatus is
And a transmitter configured to transmit a still image acquired from a camera built in the car stop block to the first information processing apparatus with the lens directed in the direction in which the parked vehicle enters .
The first information processing apparatus is
A first receiving unit that receives the still image;
A request information transmission unit that transmits request information indicating a request for a moving image to the second information processing apparatus when it is determined that there is a change based on a still image received in time series;
The second information processing apparatus is
An information processing system, comprising: a moving image transmitting unit that transmits a moving image of a time zone traced back by a predetermined time to the first information processing apparatus when the request information is received. The second information processing apparatus includes a storage unit that stores a moving image up to a predetermined time after the shooting time acquired from the camera.
The information processing system according to claim 1, wherein the moving image transmission unit transmits the moving image stored in the storage unit. The first information processing apparatus includes a recording and storing unit that associates and stores the moving image transmitted from the moving image transmitting unit, the symbol extracted from the moving image, and the second information processing apparatus that has captured the moving image. The information processing system according to claim 2. The information processing system according to claim 3, wherein the symbol is a car number displayed on a license plate of a vehicle parked at a parking location. The first information processing apparatus includes a condition determination unit that determines a parking condition of a parking place where the moving image is captured from the moving image,
The information processing system according to claim 3, wherein the determination result of the situation determination unit is recorded in the recording storage unit in association with the moving image. In a computer connected via a communication network to information processing devices respectively disposed in a plurality of vehicle stop blocks provided in a parking section ,
A reception unit that directs a lens in a direction in which a parking vehicle enters and receives a still image acquired by the information processing apparatus from a camera incorporated in the vehicle stop block ;
A request information transmission unit that transmits request information indicating a request for a moving image to the information processing apparatus when it is determined that there is a change based on a still image received in time series;
A computer, comprising: a moving image receiving unit that receives a moving image when a moving image in a time zone dated back by a predetermined time is transmitted by the information processing apparatus that has received the request information. The lens is directed in the direction in which the parked vehicle enters, and a still image acquired by a camera built in the car stop block is acquired from an information processing device respectively arranged in the car stop block ,
Determining the presence or absence of a change based on the still image acquired in time series;
When it is determined that there is a change, request information indicating a request for a moving image is transmitted to the information processing apparatus,
An information processing program that causes a computer to execute a process of receiving a moving image when a moving image in a time zone dated back by a predetermined time is transmitted by the information processing apparatus that has received the request information. The lens is directed in the direction in which the parked vehicle enters, and a still image acquired by a camera built in the car stop block is acquired from an information processing device respectively arranged in the car stop block ,
Determining the presence or absence of a change based on the still image acquired in time series;
When it is determined that there is a change, request information indicating a request for a moving image is transmitted to the information processing apparatus,
An information processing method for receiving the moving image when a moving image in a time zone dated back by a predetermined time is transmitted by the information processing apparatus having received the request information;

Images