JP7325190B2 - IMAGING DEVICE, CONTROL PROGRAM, AND OPERATION METHOD OF IMAGING DEVICE - Google Patents

Description

The present disclosure relates to imaging devices .

Japanese Patent Application Laid-Open No. 2002-200002 discloses a technique related to camera calibration.

JP 2012-10036 A

Imaging devices are desired to improve their convenience.

SUMMARY OF THE INVENTION Accordingly, it is an object of the present invention to provide a technique capable of improving the convenience of an imaging apparatus .

An imaging device , a control program and a method of operating the imaging device are disclosed. In one embodiment, an imaging device comprises a camera and a processing unit for obtaining the first image . The processing unit corrects first position information of a vehicle or pedestrian to be detected, which is captured in the first image obtained based on the installation state of the camera and the first image, based on correction information generated in advance . . The processing unit acquires second positional information of a pedestrian captured in a second image obtained by the camera before the first image based on the installation situation and the second image, and the second positional information possessed by the pedestrian. A plurality of sets associated with the third position information transmitted by the device are obtained, and the correction information is generated based on the obtained plurality of sets.

Also, in one embodiment, the control program is a control program for controlling the operation of an imaging device having a camera that obtains the first image . The control program converts the first position information of the vehicle or pedestrian to be detected, which is captured in the first image obtained based on the installation state of the camera and the first image, to the correction information generated in advance . A step of correcting based on the The control program causes the imaging device to acquire second position information of the pedestrian captured in the second image obtained by the camera before the first image, obtained based on the installation situation and the second image, and the walking A step of obtaining a plurality of sets associated with the third position information transmitted by the device possessed by the person and generating the correction information based on the obtained plurality of sets is executed.

Further, in one embodiment, the operation method of the imaging device having the camera for obtaining the first image includes the detection target vehicle captured in the first image obtained based on the installation state of the camera and the first image. Alternatively, the step of correcting the first position information of the pedestrian based on correction information generated in advance is provided. The operation method of the imaging device includes second positional information of a pedestrian captured in a second image obtained by a camera before the first image, acquired based on the installation situation and the second image, and the pedestrian obtaining a plurality of sets associated with the third position information transmitted by the device possessed by the device , and generating the correction information based on the obtained plurality of sets.

The convenience of the imaging device is improved.

1 illustrates an example of a communication system; FIG. 1 illustrates an example of a communication system; FIG. 1 is a perspective view showing an example of the appearance of a portable electronic device; FIG. It is a block diagram which shows an example of a structure of a portable electronic device. It is a block diagram which shows an example of a structure of a roadside machine. It is a block diagram which shows an example of a structure of a center apparatus. It is a flowchart which shows an example of operation|movement of a roadside machine. It is a flowchart which shows an example of operation|movement of a roadside machine. 4 is a flow chart showing an example of the operation of the center device; FIG. 4 is a diagram for explaining that position information acquired based on a camera image contains an error; It is a figure which shows an example of multiple sets of positional information sets. FIG. 4 is a diagram showing an example of a composite surface made up of a plurality of polygons; It is a flowchart which shows an example of operation|movement of a roadside machine. It is a flowchart which shows an example of operation|movement of a roadside machine. It is a flowchart which shows an example of operation|movement of a roadside machine.

<Outline of communication system>
1 and 2 are diagrams showing an example of a communication system 1. FIG. The communication system 1 is, for example, a system used in Intelligent Transport Systems (ITS). Specifically, the communication system 1 can be used as an ITS safe driving support communication system. A driving safety support communication system is called a driving safety support system or a driving safety support radio system.

As shown in FIG. 1, a communication system 1 includes a plurality of portable electronic devices 2, a plurality of roadside units 3, a plurality of electronic devices 40 mounted on a plurality of vehicles 4, and a center device 5. Prepare. A plurality of portable electronic devices 2 , a plurality of roadside devices 3 , a plurality of electronic devices 40 and a center device 5 are connected to a network 6 . A plurality of portable electronic devices 2 , a plurality of roadside devices 3 , a plurality of electronic devices 40 , and the center device 5 can communicate with each other through the network 6 . The vehicle 4 is capable of communicating with other devices, such as the portable electronic device 2, by means of electronic equipment 40 mounted thereon. The network 6 includes relay devices, the Internet, and the like. Hereinafter, the portable electronic device 2 may be simply referred to as the electronic device 2 in some cases. Also, the electronic device 40 mounted on the vehicle 4 may be called an in-vehicle device 40 .

In the communication system 1, as shown in FIG. 2, a roadside unit 3 arranged at an intersection 7 or the like, a vehicle 4 such as an automobile running on a roadway 8, and an electronic device 2 possessed by a user 20 walking on a sidewalk 9. can communicate with each other. It can also be said that the user 20 is a pedestrian. In addition, multiple vehicles 4 can communicate with each other. The communication between the roadside device 3 and the vehicle 4, the communication between the vehicles 4, the communication between the roadside device 3 and the pedestrian's electronic device 2, and the communication between the pedestrian's electronic device 2 and the vehicle 4 are respectively , road-to-vehicle communication, vehicle-to-vehicle communication, road-to-foot communication, and pedestrian-to-vehicle communication. Henceforth, the user 20 may be called the pedestrian 20. FIG.

The electronic device 2 includes operation buttons 220, which will be described later. When the operation button 220 is operated, the electronic device 2 can transmit the position information of the electronic device 2 and the like to the center device 5 . The electronic device 2 carried by the pedestrian 20 is sometimes called a walking terminal.

The roadside device 3 can notify the vehicle 4 and the electronic device 2 of, for example, information regarding lighting of the traffic signal 10 and information regarding road regulations. In addition, the roadside unit 3 can detect vehicles 4 and pedestrians around it. The roadside unit 3 placed at the intersection 7 can detect pedestrians crossing the pedestrian crossing 80, for example. Then, the roadside device 3 can notify the vehicle 4 and the electronic device 2 of the detected information regarding the vehicle 4 and the pedestrian. Further, the roadside device 3 can notify other vehicles 4 and electronic devices 2 of information notified from the vehicles 4 and the electronic devices 2 .

Here, in this example, the sidewalk is a road for pedestrians to pass along the roadway. Pedestrian crossings that cross the roadway are separate from sidewalks and are included in the roadway.

The vehicle 4 can notify the other vehicles 4, the roadside device 3, and the electronic device 2 of its own position, speed, information about the blinker, and the like using the in-vehicle device 40 . The vehicle 4 can assist the driver in driving safely by giving various notifications such as warnings to the driver based on information notified from other devices. The vehicle 4 can make various notifications to the driver using the speaker, the display device, and the like provided in the vehicle-mounted device 40 . The in-vehicle device 40 is, for example, a car navigation device having a communication function.

Thus, the communication system 1 supports the safe driving of the driver of the vehicle 4 by performing road-to-vehicle communication, vehicle-to-vehicle communication, road-to-walk communication, and pedestrian-to-vehicle communication.

In the example of FIG. 2, an automobile vehicle is shown as the vehicle 4, but the vehicle 4 may be a vehicle other than an automobile. For example, the vehicle 4 may be a bus vehicle or a tram vehicle.

<Configuration example of electronic device>
FIG. 3 is a perspective view showing an example of the appearance of the electronic device 2. As shown in FIG. FIG. 4 is a block diagram showing an example of the electrical configuration of the electronic device 2. As shown in FIG. As shown in FIG. 3 , the electronic device 2 includes a case 21 that forms an exterior of the electronic device 2 . The case 21 has, for example, an elongated plate shape, and is shaped so that the user 20 can hold it with one hand. Note that the shape of the case 21 is not limited to the example shown in FIG.

The electronic device 2 includes, in addition to the case 21, a control section 200, a communication section 210, an operation button 220, and a satellite signal receiving section 230, as shown in FIG. The control unit 200 , communication unit 210 , operation buttons 220 and satellite signal reception unit 230 are housed inside the case 21 . A part of the surface of the operation button 220 is exposed from the case 21 so that the user 20 can operate the operation button 220 .

The control unit 200 can comprehensively manage the operation of the electronic device 2 by controlling other components of the electronic device 2 . The control unit 200 can also be called a control device or a control circuit. Control unit 200 includes at least one processor to provide control and processing power to perform various functions, as described in further detail below.

According to various embodiments, the at least one processor may be implemented as a single integrated circuit (IC) or as a plurality of communicatively connected integrated circuits IC and/or discrete circuits. good. The at least one processor can be implemented according to various known techniques.

In one embodiment, a processor includes one or more circuits or units configured to perform one or more data computing procedures or processes, such as by executing instructions stored in associated memory. In other embodiments, the processor may be firmware (eg, discrete logic components) configured to perform one or more data computing procedures or processes.

According to various embodiments, the processor is one or more processors, controllers, microprocessors, microcontrollers, application specific integrated circuits (ASICs), digital signal processors, programmable logic devices, field programmable gate arrays, or any of these. Any combination of devices or configurations, or other known combinations of devices and configurations, may be included to perform the functions described below.

In this example, the control unit 200 includes a CPU (Central Processing Unit) 201 and a storage unit 202 . The storage unit 202 includes non-temporary recording media readable by the CPU 201, such as ROM (Read Only Memory) and RAM (Random Access Memory). The ROM included in the storage unit 202 is, for example, a flash ROM (flash memory) that is nonvolatile memory. The storage unit 202 stores a control program 202a for controlling the electronic device 2 and the like. Various functions of the control unit 200 are realized by the CPU 201 executing a control program 202a in the storage unit 202. FIG.

Note that the configuration of the control unit 200 is not limited to the above example. For example, the control unit 200 may include multiple CPUs 201 . Also, the control unit 200 may include at least one DSP (Digital Signal Processor). Also, all the functions of the control unit 200 or part of the functions of the control unit 200 may be implemented by hardware circuits that do not require software to implement the functions. The storage unit 202 may also include a non-temporary computer-readable recording medium other than the ROM and RAM. The storage unit 202 may include, for example, a small hard disk drive and an SSD (Solid State Drive).

The communication unit 210 is connected to the network 6 at least either wiredly or wirelessly. The communication unit 210 of the electronic device 2 can communicate with other electronic devices 2 , the roadside device 3 , the vehicle-mounted device 40 and the center device 5 through the network 6 . Communication unit 210 outputs information received from another device to control unit 200 . Communication unit 210 transmits information from control unit 200 to other devices. The communication unit 210 can also be said to be a communication circuit.

Note that the communication unit 210 of the electronic device 2 may be capable of directly communicating with at least one of the other electronic device 2, the roadside device 3, the vehicle-mounted device 40, and the center device 5 without going through the network 6. . In this case, the communication unit 210 performs direct wireless communication with at least one of the other electronic device 2, the roadside device 3, the vehicle-mounted device 40, and the center device 5 using, for example, the 700 MHz band assigned to the ITS. good too. Also, the communication unit 210 may be capable of wirelessly communicating with a base station of a mobile phone system. In this case, the communication unit 210 can communicate with devices connected to the Internet (for example, mobile phones, web servers, etc.) through the base station. Also, the communication unit 210 may be capable of performing short-range wireless communication. For example, the communication unit 210 may be capable of wireless communication conforming to Bluetooth (registered trademark). The communication unit 210 may be capable of wireless communication using a wireless LAN (Local Area Network) such as WiFi. Also, the communication unit 210 may be capable of wireless communication based on another wireless communication scheme.

The satellite signal receiving unit 230 can receive satellite signals transmitted by positioning satellites. The satellite signal reception unit 230 can also be said to be a satellite signal reception circuit. The satellite signal reception unit 230 can acquire position information indicating the absolute position of the electronic device 2 (in other words, the position in the absolute coordinate system) based on the received satellite signal. It can be said that the receiving unit 230 is a position acquisition unit or a position acquisition circuit that acquires the position information of the electronic device 2 . The position information acquired by the satellite signal receiving unit 230 is represented by latitude, longitude and altitude, for example. The position information of the electronic device 2 can also be said to be position information indicating the position of the user 20 (in other words, the pedestrian 20) holding the electronic device 2. FIG. Hereinafter, the satellite signal receiving section 230 may be simply referred to as the receiving section 230 in some cases.

The receiving unit 230 is, for example, a GPS receiver, and can receive radio signals from GPS (Global Positioning System) positioning satellites. The receiving unit 230 calculates the current position of the electronic device 2 by, for example, latitude, longitude and altitude based on the received radio signal, and outputs position information including the calculated latitude, longitude and altitude to the control unit 200 . The receiving unit 230 repeatedly acquires position information.

Note that the receiving unit 230 may obtain the position information of the electronic device 2 based on signals from positioning satellites of GNSS (Global Navigation Satellite System) other than GPS. For example, the receiving unit 230 receives signals from positioning satellites such as GLONASS (Global Navigation Satellite System), IRNSS (Indian Regional Navigational Satellite System), COMPASS, Galileo, or Quasi-Zenith Satellites System (QZSS). The position information of the electronic device 2 may be obtained.

The operation button 220 is, for example, a push button. When the operation button 220 is operated, the control unit 200 generates, for example, operation notification information that notifies that the operation button 220 has been operated. Then, the control unit 200 causes the communication unit 210 to transmit the generated operation notification information to the center device 5, for example. The operation notification information includes, for example, position information acquired by the receiving unit 230 and identification information of the electronic device 2 .

The configuration of the electronic device 2 is not limited to the above example. The electronic device 2 may be, for example, a mobile phone such as a smart phone. Also, the electronic device 2 may be, for example, a tablet terminal, a personal computer, a wearable device, or the like. The wearable device employed as the electronic device 2 may be a type worn on the arm such as a wristband type or a wristwatch type, or may be a type worn on the head such as a headband type or glasses type. , a type that is worn on the body, such as a clothing type.

<Configuration example of roadside unit>
FIG. 5 is a block diagram showing an example of the configuration of the roadside unit 3. As shown in FIG. As shown in FIG. 5 , the roadside device 3 includes a control section 300 , a communication section 310 , a camera 320 and a sensor 330 . The control unit 300 can comprehensively manage the operation of the roadside device 3 by controlling other components of the roadside device 3 . The control unit 300 can also be called a control device or a control circuit. Control unit 300 includes at least one processor to provide control and processing power to perform various functions, as described in further detail below. The above description of the processor included in the controller 200 of the electronic device 2 can also be applied to the processor included in the controller 300 .

In this example, the control unit 300 includes a CPU 301 and a storage unit 302 . The storage unit 302 includes a non-temporary recording medium readable by the CPU 301, such as ROM and RAM. The ROM included in the storage unit 302 is, for example, a flash ROM that is nonvolatile memory. The storage unit 302 stores a control program 302a for controlling the roadside device 3 and the like. Various functions of the control unit 300 are implemented by the CPU 301 executing a control program 302 a in the storage unit 302 .

Note that the configuration of the control unit 300 is not limited to the above example. For example, the control unit 300 may include multiple CPUs 301 . Also, the control unit 300 may include at least one DSP. Further, all functions of the control unit 300 or some functions of the control unit 300 may be realized by a hardware circuit that does not require software to realize the functions. The storage unit 302 may also include a non-temporary computer-readable recording medium other than the ROM and RAM.

The communication unit 310 is connected to the network 6 at least either wiredly or wirelessly. The communication unit 310 of the roadside device 3 can communicate with the other roadside device 3 , the electronic device 2 , the vehicle-mounted device 40 and the center device 5 through the network 6 . Further, the communication unit 310 can directly wirelessly communicate with the in-vehicle device 40 without going through the network 6 . The communication unit 310 can perform direct wireless communication with the vehicle-mounted device 40 around the roadside device 3, for example, using the 700 MHz band assigned to the ITS. Communication unit 310 outputs information received from another device to control unit 300 . Communication unit 310 transmits information from control unit 300 to other devices. The communication unit 310 can also be said to be a communication circuit.

Note that the communication unit 310 of the roadside device 3 may be able to directly communicate with other roadside devices 3 , electronic devices 2 and the center device 5 without going through the network 6 . In this case, the communication unit 310 may perform direct wireless communication with other roadside devices 3, electronic devices 2, and center device 5 using, for example, the 700 MHz band assigned to ITS. Also, the communication unit 310 may be capable of wireless communication with a base station of a mobile phone system. Also, the communication unit 310 may be capable of performing short-range wireless communication. Also, the communication unit 310 may be capable of performing wireless communication using a wireless LAN such as WiFi. Also, the communication unit 310 may be capable of wireless communication based on another wireless communication scheme.

The camera 320 can photograph the state of the road around the roadside unit 3 . An image generated by camera 320 is input to control unit 300 . The camera 320 can shoot moving images, for example.

The sensor 330 can detect road conditions around the roadside unit 3 . Sensor 330 includes, for example, a radar sensor and an infrared sensor. Sensors 330 may, for example, detect pedestrians and vehicles 4 in the vicinity of roadside units 3 . Sensor 330 outputs the detection result to control unit 300 .

<Configuration example of center device>
FIG. 6 is a block diagram showing an example of the configuration of the center device 5. As shown in FIG. As shown in FIG. 6 , the center device 5 has a control section 500 and a communication section 510 . The control unit 500 can control the operation of the center device 5 by controlling other components of the center device 5 . The control unit 500 can also be called a control device or a control circuit. Control unit 500 includes at least one processor to provide control and processing power to perform various functions, as described in further detail below. The above description of the processor included in the controller 200 of the electronic device 2 can also be applied to the processor included in the controller 500 .

In this example, the control unit 500 includes a CPU 501 and a storage unit 502 . The storage unit 502 includes non-temporary recording media readable by the CPU 501, such as ROM and RAM. The ROM included in the storage unit 502 is, for example, a flash ROM that is nonvolatile memory. The storage unit 502 stores a control program 502a for controlling the center device 5 and the like. Various functions of the control unit 500 are realized by the CPU 501 executing a control program 502 a in the storage unit 502 .

Note that the configuration of the control unit 500 is not limited to the above example. For example, the control unit 500 may include multiple CPUs 501 . Also, the control unit 500 may include at least one DSP. Further, all functions of the control unit 500 or part of the functions of the control unit 500 may be implemented by hardware circuits that do not require software to implement the functions. The storage unit 502 may also include a non-temporary computer-readable recording medium other than the ROM and RAM.

The communication unit 510 is connected to the network 6 at least either wiredly or wirelessly. The communication unit 510 can communicate with the electronic device 2 , the roadside device 3 and the in-vehicle device 40 through the network 6 . Communication unit 510 outputs information received from another device to control unit 500 . Communication unit 510 transmits information from control unit 500 to another device. The communication unit 510 can also be said to be a communication circuit.

Note that the communication unit 510 may be capable of wireless communication with a base station of the mobile phone system. Also, the communication unit 510 may be capable of performing short-range wireless communication. Also, the communication unit 510 may be capable of performing wireless communication using a wireless LAN such as WiFi. Communication unit 510 may be capable of wireless communication based on another wireless communication scheme.

The center device 5 configured as described above executes predetermined processing upon receiving the operation notification information from the electronic device 2 . For example, consider a case where the operation button 220 is a button that the user 20 operates when trouble occurs in the user 20 . A child or an elderly person possessing the electronic device 2 operates the operation button 220 when, for example, they are attacked by a suspicious person, suddenly become ill and cannot move, or are involved in a traffic accident. In the center device 5 , when the communication section 510 receives the operation notification information from the electronic device 2 , the control section 500 determines that the user 20 of the electronic device 2 is in trouble. Then, the control unit 500 performs processing corresponding to the trouble that has occurred. For example, the control unit 500 generates trouble notification information that notifies that trouble has occurred at the position indicated by the position information included in the received operation notification information. Then, control unit 500 causes communication unit 510 to transmit the generated trouble notification information to a predetermined communication device. The communication unit 510 may transmit the trouble notification information to a communication device owned by, for example, a police station or a police box. At this time, the communication unit 510 may transmit the trouble notification information to the communication device at the police station or police box closest to the location of the user 20 in trouble. When a communication device provided at a police station or a police box receives trouble notification information, for example, a police officer rushes to the place where the troubled user 20 exists. Further, the communication unit 510 may transmit trouble notification information to a communication device owned by a security company. When the communication device provided by the security company receives the trouble notification information, for example, a security guard rushes to the place where the troubled user 20 exists. In addition, when a notification destination device is associated in advance with identification information included in the received operation notification information, communication unit 510 may transmit trouble notification information to the notification destination device. . If the user 20 is a child, the notification destination device may be, for example, the mobile phone of the user 20's parent. Also, if the user 20 is an elderly person, the mobile phone of the child of the user 20 can be considered as the notification destination device, for example.

Note that the user 20 may operate the operation button 220 in cases other than when trouble occurs. Further, the operation of the center device 5 when the center device 5 receives the operation notification information is not limited to the above.

<Operation example of communication system>
FIG. 7 is a flow chart showing an example of the operation of the roadside unit 3 in operation. The roadside device 3 repeats the processing shown in FIG. 7, for example. As shown in FIG. 7, in step s1, the control section 300 acquires the latest image obtained by the camera 320 from the camera 320 concerned. Hereinafter, an image obtained by the camera 320 may be called a camera image.

Next, in step s2, the control unit 300 determines whether or not the target object to be detected is captured in the camera image obtained in step s1. If it is determined that the target object is captured in the camera image, step s3 is executed. On the other hand, if the target object is not captured in the camera image, the processing in FIG. 7 ends. A target object is, for example, an object in contact with the ground. The target object is, for example, a pedestrian on the road or the vehicle 4 on the road. In step s2, the control unit 300 can determine whether or not the target object is captured in the camera image by performing image processing on the camera image. The control unit 300 may use artificial intelligence (AI), for example, to determine whether or not the target object is captured in the camera image. For example, the control unit 300 may use machine learning using a neural network to determine whether or not the target object is captured in the camera image.

If YES is determined in step s2, in step s3, the control unit 300 acquires position information indicating the absolute position of the target object based on the camera image obtained in step s1. This position information is represented by, for example, latitude, longitude, and altitude, like the position information acquired by the receiver 230 of the electronic device 2 . In step s3, the control unit 300 identifies a region of the ground where the part in contact with the target object is captured in the camera image that captures the target object. By performing image processing on the camera image, the control unit 300 can specify an area of the ground where the part in contact with the target object is captured in the camera image. The control unit 300 may, for example, use artificial intelligence such as machine learning using a neural network to identify a region of the ground where the target object is in contact with in the camera image. Then, the control unit 300 acquires, as the position information of the target object, the position information indicating the absolute position of the place in the specified area, that is, the position on the ground where the target object is in contact.

For example, consider the case where the target object is a pedestrian. In this case, the control unit 300 specifies a region of the ground in which a portion of the ground with which the foot of the pedestrian is in contact is captured in the camera image of the pedestrian. Then, the control unit 300 acquires, as the pedestrian's position information, position information indicating the absolute position of the place in the specified area, that is, the position on the ground where the foot of the pedestrian is in contact.

As another example, consider the case where the target object is a vehicle 4 . In this case, the control unit 300 specifies a region of the ground where the portion of the ground where the tires or wheels of the vehicle 4 are in contact is captured in the camera image that captures the vehicle 4 . Then, the control unit 300 acquires, as the position information of the vehicle 4, the position information indicating the absolute position of the position in the specified area, that is, the position on the ground where the tires or wheels of the vehicle 4 are in contact.

Here, in this example, the angle of view, the installation height, and the installation angle (in other words, the orientation of the camera 320) of the camera 320 of the roadside device 3 are constant. Therefore, when the ground captured in the camera image is horizontal, the control unit 300 of the roadside device 3 controls the angle of view, the installation height, and the installation angle of the camera 320 of the roadside device 3, and the absolute position of the roadside device 3. Based on the position information indicating (latitude, longitude and altitude), the absolute position (latitude, longitude and altitude) of each place on the ground captured in the camera image can be obtained. When obtaining the position of a certain place on the ground, the control unit 300 determines the distance from the camera 320 to the certain place and the distance from the camera 320 to the certain place based on the angle of view, the installation height and the installation angle of the camera 320. Identify direction. Then, the control unit 300 determines the absolute position (latitude, longitude, and altitude). The storage unit 302 of the roadside device 3 stores position information indicating the absolute position of the roadside device 3 . In this example, in step s3, the control unit 300 obtains the position information of the target object based on the camera image, assuming that the ground captured in the camera image is horizontal. It can be said that the position information of the target object acquired in step s3 is the position information of the target object in contact with the ground assuming that the ground is horizontal.

Next, at step s4, the control unit 300 corrects the position information obtained at step s3 based on the correction information. This provides the position information of the target object in contact with the actual ground rather than the ground that is assumed to be flat. The correction information will be explained later in detail.

Next, in step s5, the control unit 300 executes processing based on the corrected position information obtained in step s4. For example, consider the case where the target object is a pedestrian. In this case, the control unit 300 determines whether the pedestrian is on the roadway 8 or on the sidewalk 9 based on the corrected position information of the pedestrian and the map information in the storage unit 302, for example. judge. When determining that a pedestrian exists on the roadway 8 , the control unit 300 generates pedestrian notification information for notifying that a pedestrian exists on the roadway 8 . Then, the control unit 300 causes the communication unit 310 to notify the surrounding vehicles 4 of the generated pedestrian notification information. At this time, the communication unit 310 broadcasts the pedestrian notification information to the surrounding vehicles 4 using, for example, the 700 MHz band assigned to the ITS. Upon receiving the pedestrian notification information, the vehicle-mounted device 40 of the vehicle 4 notifies the driver of the presence of a pedestrian on the surrounding roadway 8 using a speaker, a display device, or the like.

As another example, consider the case where the target object is a vehicle 4 . In this case, the control unit 300 determines whether or not the vehicle 4 has been stopped near the intersection 7 for a predetermined time or longer based on the corrected position information of the vehicle 4 and the map information in the storage unit 302, for example. judge. At this time, based on the corrected position information of the vehicle 4 obtained in step s4 and the corrected position information of the vehicle 4 obtained in the past, the control unit 300 determines whether the vehicle 4 has stopped for a predetermined time or longer. Determine whether or not When the vehicle 4 is stopped, the plurality of pieces of corrected position information acquired by the control unit 300 do not change. When the control unit 300 determines that the vehicle 4 has been parked near the intersection 7 for a predetermined time or longer, it generates stopped vehicle notification information for notifying that the vehicle 4 that has been parked for a long time exists near the intersection 7. do. Then, the control unit 300 causes the communication unit 310 to notify the surrounding vehicles 4 of the generated stopped vehicle notification information. At this time, the communication unit 310 broadcasts the stopped vehicle notification information to the surrounding vehicles 4 using, for example, the 700 MHz band assigned to the ITS. When receiving the stopped vehicle notification information, the in-vehicle device 40 of the vehicle 4 notifies the driver that the vehicle 4 that has been stopped for a long time is near the intersection 7 by using a speaker, a display device, or the like. to notify you.

Note that the target object is not limited to the above examples. Further, the processing based on the post-correction position information in step s5 is not limited to the above example.

<Example of how to generate correction information>
Next, a method for generating correction information used in step s4 will be described. The correction information is generated by the roadside device 3 . FIG. 8 is a flow chart showing an example of correction information generation processing in the roadside device 3 . The correction information generation process shown in FIG. 8 is executed, for example, by the roadside unit 3 immediately after installation and before operation. Note that the correction information generation process may be executed by the roadside device 3 in operation. Hereinafter, the roadside device 3 to be described may be referred to as the target roadside device 3 .

As shown in FIG. 8, in step s1, the control unit 300 of the target roadside device 3 generates first request information requesting the location information of the user 20 of the electronic device 2 located near the target roadside device 3. do. The first request information includes position information indicating the absolute position of the target roadside device 3 . The control unit 300 causes the communication unit 310 to transmit the generated first request information to the center device 5 .

In the center device 5, as shown in FIG. 9, when the communication unit 510 receives the first request signal from the target roadside device 3 in step s21, the control unit 500 places the electronic device near the target roadside device 3 in step s22. Determine whether or not the device 2 exists.

Here, each electronic device 2 regularly or irregularly transmits the latest position information obtained by the receiving unit 230 and the identification information of the electronic device 2 to the center device 5 . The center device 5 associates the position information and the identification information transmitted from the electronic device 2 with each other and stores them in the storage unit 502 . When receiving the new position information from the electronic device 2, the center device 5 replaces the position information associated with the identification information of the electronic device 2 with the newly received position information in the storage unit 502, The location information of the electronic device 2 is updated.

In step s22, the control unit 500 compares the position information included in the first request signal (that is, the position information of the target roadside device 3) with the position information of each electronic device 2 in the storage unit 502, and the comparison is performed. Based on the result, it is determined whether or not the electronic device 2 exists near the target roadside device 3 . In step s22, the control unit 500 determines the distance between the position indicated by the position information of each electronic device 2 in the storage unit 502 and the position indicated by the position information included in the first request signal. Determine whether or not the value is equal to or less than the threshold value. Then, if the distance between the position indicated by the position information of the electronic device 2 and the position indicated by the position information included in the first request signal is equal to or less than the threshold value, the control unit 500 It is determined that the electronic device 2 is present in .

When the electronic device 2 exists near the target roadside device 3, there is a high possibility that the electronic device 2 will appear in the image obtained by the camera 320 of the target roadside device 3. Therefore, when the electronic device 2 exists near the target roadside device 3 , it can be said that the target roadside device 3 is the roadside device 3 with a high possibility that the electronic device 2 appears in the image obtained by the camera 320 . Therefore, the process of step s22 can be said to be a process of determining whether or not there is a high possibility that the electronic device 2 will appear in the image obtained by the camera 320 of the target roadside device 3 . The threshold value used in step s22 is determined, for example, according to the imaging range of the camera 320 of the target roadside device 3. Hereinafter, the electronic device 2 existing near the target roadside device 3 may be referred to as the target electronic device 2 .

If NO is determined in step s22, the control unit 500 executes time elapse processing in step s23. In other words, the control unit 500 waits for the lapse of a certain period of time. After a certain period of time has passed, step s22 is executed again. Thereafter, the center device 5 operates similarly.

On the other hand, if YES is determined in step s22, step s24 is executed. At step s<b>24 , the control unit 500 generates second request information requesting the location information of the user 20 of the target electronic device 2 . Then, the control unit 300 causes the communication unit 310 to transmit the generated second request information to the target electronic device 2 . Henceforth, the user 20 of the target electronic device 2 may be called the target user 20 or the target pedestrian 20. FIG.

In the target electronic device 2 , when the communication section 210 receives the second request information, the control section 200 causes the communication section 210 to transmit the latest position information obtained by the reception section 230 to the center device 5 .

After step s24, the communication unit 510 of the center device 5 receives the position information of the target user 20 in step s25. The position information received by the communication unit 510 is input to the control unit 500 . After step s25, the control unit 500 causes the communication unit 510 to transmit the received location information of the target user 20 to the target roadside device 3 at step s26.

Returning to FIG. 8, after step s11, the communication unit 310 of the target roadside device 3 receives the position information of the target user 20 from the center device 5 in step s12. After that, in step s13, the control unit 300 executes user identification processing for identifying the target user 20 in the camera image obtained by the camera 320 based on the received position information of the target user 20. FIG. The user identification process can also be said to be a process of determining whether or not the target user 20 is captured in the camera image. User identification processing will be described later in detail.

Next, in step s14, the control section 300 determines whether or not the target user 20 has been identified in the camera image in the user identification processing of step s13. When the target user 20 is specified in the user specifying process, in other words, when the target user 20 is captured in the camera image, step s15 is executed. On the other hand, if the target user 20 is not specified in the user specifying process, in other words, if the target user 20 is not captured in the camera image, step s11 is executed again. Thereafter, the target roadside device 3 operates in the same manner.

In step s15, the control unit 300 acquires position information indicating the absolute position of the target user 20 based on the camera image in which the target user 20 is captured. The processing of step s15 is the same as the processing of step s3 (see FIG. 7) described above when the target object is a pedestrian. The positional information acquired in step s15 is the positional information of the target user 20 in contact with the ground assuming that the ground is flat, like the positional information acquired in step s3.

Hereinafter, the positional information of the target user 20 received by the target roadside device 3 from the center device 5 in step s12 may be referred to as first positional information. Further, in step s15, the target user's position information acquired by the target roadside device 3 based on the camera image may be referred to as second position information. Since the second position information is the position information of the place where the target user's foot touches on the ground captured in the camera image, it can be said that the second position information is the position information of the ground location captured in the camera image.

After step s15, in step s16, the control unit 300 stores the first location information obtained in step s12 and the second location information obtained in step s15 as a location information set of the target user 20 in the storage unit 302. memorize to

Next, in step s17, the control section 300 determines whether or not the necessary number of sets of position information have been obtained. In other words, the control unit 300 determines whether or not the necessary number of sets of position information have been stored in the storage unit 302 . For example, the control unit 300 obtains a predetermined number of sets of position information or more, and the position indicated by a plurality of pieces of second position information included in the obtained plurality of sets of position information is the ground surface captured in the camera image. , we determine that we have obtained as many sets of location information as we need. In other words, when a predetermined number of sets of position information or more are obtained, and the position information of most of the ground captured in the camera image is obtained, the control unit 300 determines that the necessary number of position information sets have been obtained. judge. When the pedestrian 20 carrying the electronic device 2 moves for a while in the shooting range of the camera 320 of the target roadside device 3, the target roadside device 3 can acquire a plurality of sets of position information. Further, even when there are a plurality of pedestrians 20 carrying electronic devices 2 in the shooting range of the camera 320 of the target roadside device 3, the target roadside device 3 can acquire a plurality of sets of position information. .

If NO is determined in step s17, step s11 is executed again, and thereafter the target roadside device 3 operates in the same manner. On the other hand, if YES is determined in step s17, step s18 is executed. N represents the number of sets of position information stored in the storage unit 302 when YES is determined in step s17.

At step s<b>18 , the control unit 300 generates correction information based on the N pairs of position information stored in the storage unit 302 . Then, the control unit 300 stores the generated correction information in the storage unit 302 . In step s18, the control unit 300 generates gradient information indicating the gradient of the ground captured in the camera image, based on the N sets of position information, for example. The controller 300 then uses the generated gradient information as correction information. At step s4 in FIG. 7, the target roadside device 3 corrects the position information obtained at step s3 based on the gradient information.

Here, even if a person is captured in the same area of the camera image (in other words, even if the position of the person's image in the camera image is the same), the ground is horizontal and sloped. The absolute position of the person (in other words, the position of the person in real space) differs depending on the situation. The positional information of the target object acquired based on the camera image in step s3 described above is the positional information of the target object in contact with the ground, assuming that the ground is horizontal. Therefore, if the ground actually has a slope, the position information of the target object obtained based on the camera image contains an error.

FIG. 10 is a diagram for explaining errors included in the position information of the target object acquired based on the camera image. In FIG. 10, the actual ground surface with a slope is shown as ground surface 600 and the ground surface assumed to be horizontal is shown as ground surface 610 . FIG. 10 also shows the position 640 of the target object 620 obtained by the control unit 300 based on the camera image obtained by the camera 320 . And in FIG. 10 the actual position 630 of the target object 620 is shown. When the target object 620 is the pedestrian 20 , the position indicated by the position information obtained by the receiver 230 of the electronic device 2 possessed by the pedestrian 20 is the actual position 630 of the target object 620 .

As shown in FIG. 10, the position 640 of the target object 620 determined based on the camera image is different from the actual position 630 of the target object 620 when the ground has a slope. Therefore, if the ground has a slope, the position information of the target object obtained based on the camera image contains an error.

Thus, the position information of the target object obtained based on the camera image contains an error due to the gradient of the ground captured in the camera image. Therefore, in this example, the roadside unit 3 generates gradient information indicating the gradient of the ground captured in the camera image, and corrects the position information of the target object acquired based on the camera image based on this gradient information. This makes it possible to obtain highly accurate position information about the target object. An example of a method for generating gradient information will be described below. In the following description, an XYZ coordinate system is used, which consists of an X-axis representing one of latitude and longitude, a Y-axis representing the other of latitude and longitude, and a Z-axis representing altitude. Hereinafter, simply speaking of an XYZ coordinate system means an XYZ coordinate system consisting of an X axis representing one of latitude and longitude, a Y axis representing the other of latitude and longitude, and a Z axis representing altitude.

FIG. 11 is a diagram showing an example of N sets of position information sets used in generating slope information. In FIG. 11, the position 710 indicated by the first positional information included in the positional information set is indicated by a white circle. A position 710 is the position indicated by the position information acquired by the receiver 230 of the electronic device 2 and indicates the actual position of the user 20 . Further, in FIG. 11, the position 720 indicated by the second positional information included in the positional information set is indicated by a black circle. A position 720 is a position indicated by the positional information of the user 20 acquired based on the camera image when it is assumed that the ground captured in the camera image is horizontal. In FIG. 11, positions 710 and 720 indicated by the first and second position information constituting each position information group are connected with each other by a dashed line. Hereinafter, the position 720 corresponding to the position 710 means the position 720 connected to the position 710 by a dashed line.

As described above, the second positional information is positional information indicating the position of a place on the ground, assuming that the ground is horizontal. Accordingly, the positions 720 indicated by the N pieces of second position information included in the N sets of position information are on the same plane 800 as shown in FIG. 11 . Henceforth, this plane 800 may be called a reference plane 800 . Reference plane 800 can be said to represent the ground, which is assumed to be horizontal.

In this example, in step s18, the control unit 300 places the positions 710 indicated by the N pieces of first position information included in the N sets of position information and the N positions included in the N sets of position information in the XYZ coordinate system. The position 720 indicated by the second position information is arranged. Then, the control unit 300 connects the N positions 710 arranged in the XYZ coordinate system with a line 910 to form a plurality of continuously arranged polygons 920 in the XYZ coordinate system. The control unit 300 forms one polygon 920 by connecting a plurality of positions 710 corresponding to a plurality of positions 720 close to each other with a line 910 . Then, the control unit 300 uses the composite surface information representing the composite surface 930 composed of the plurality of polygons 920 as gradient information representing the slope of the ground. It can be said that the composite surface 930 represents the actual ground surface.

FIG. 12 is a diagram illustrating an example of a composite surface 930 formed from multiple locations 710 shown in FIG. In the example of FIG. 12, polygon 920 is triangular. The control unit 300 forms one triangle by connecting the three positions 710 corresponding to the three positions 720 close to each other with a line 910 .

At step s4 in FIG. 7, the control unit 300 corrects the position information acquired at step s3 based on the composite surface information. At step s4, for example, the control unit 300 sets a reference plane 800 including N positions 720 in the XYZ coordinate system. Then, the control unit 300 arranges the position indicated by the position information of the target object acquired based on the camera image as the target position. The target position is located within the reference plane 800 . Then, in the XYZ coordinate system, the control unit 300 obtains an intersection point between a line passing through the target position and perpendicular to the reference plane 800 and the combined plane 930 indicated by the combined plane information. The control unit 300 adopts the position (that is, latitude, longitude and altitude) of the obtained intersection in the XYZ coordinate system as the position indicated by the corrected position information. As a result, the position information acquired in step s3 is corrected based on the composite surface 930. FIG.

Note that the correction information is not limited to the above examples. Also, the method of correcting the position information based on the correction information in step s4 is not limited to the above example. For example, N sets of location information sets may be used as correction information. In this case, in step s4, the control unit 300, for example, among the positions 720 indicated by the N pieces of second position information included in the N sets of position information, the position indicated by the position information acquired in step s3 is the closest. Identify nearby locations 720 . Then, the control unit 300 adopts, for example, the position 710 corresponding to the specified position 720 as the position indicated by the corrected position information.

As described above, in the present example, the control unit 300 of the roadside unit 3 acquires the position information indicating the position of the pedestrian 20, which is acquired based on the camera image of the pedestrian 20 carrying the electronic device 2, A plurality of sets of position information including the position information indicating the position of the pedestrian 20 transmitted by the electronic device 2 are obtained. Then, the control unit 300 generates correction information based on the acquired plural sets of position information. As a result, the roadside device 3 can generate the correction information when the pedestrian 20 carrying the electronic device 2 is present in the shooting range of the camera 320 . Therefore, the roadside device 3 can easily generate the correction information. As a result, the convenience of the roadside device 3 is improved.

Also, in this example, the control unit 300 generates gradient information based on multiple sets of position information. On the other hand, gradient information can also be acquired by actually surveying the gradient of the ground. In this case, the work load is heavy due to reasons such as the need to stop traffic during surveying. On the other hand, in this example, since the control unit 300 generates gradient information based on multiple sets of position information, the roadside device 3 can easily generate gradient information.

13 and 14 are flow charts showing an example of details of the above-described user identification process (step s13 in FIG. 8). As shown in FIG. 13, in step s41, the control unit 300 executes person identification processing for identifying a person in the camera image. The person identification process can also be said to be a process of determining whether or not a person is captured in the camera image. In step s41, the control unit 300 can identify a person in the camera image by performing image processing on the camera image. The control unit 300 may identify a person in the camera image, for example, using artificial intelligence such as machine learning using a neural network.

After the person identifying process is executed, in step s42, the control section 300 determines whether or not the person has been identified in the camera image in the person identifying process. If the person cannot be identified in the camera image, in other words, if the person is not captured in the camera image, the control unit 300 terminates the user identification process. In this case, in step s14 of FIG. 8 described above, it is determined that the target user 20 could not be specified in the camera image.

On the other hand, when the control unit 300 determines in step s42 that the person has been identified in the camera image, in other words, when the person appears in the camera image, step s43 is executed. In step s43, the control unit 300 determines whether or not the person specified in the person specifying process (hereinafter sometimes referred to as a specified person) is one person. In other words, the control unit 300 determines whether or not there is only one person in the camera image.

If it is determined in step s43 that there is only one specific person, in step s44, the control unit 300, in the same manner as in steps s3 and s15, determines the absolute number of the specific person based on the camera image of the specific person. Get location information that indicates a specific location. This position information is represented by, for example, latitude, longitude, and altitude, like the position information acquired by the receiver 230 of the electronic device 2 .

After step s44, in step s45, the control unit 300 determines whether or not the position of the target user 20 and the position of the specific person are close. In step s45, the control unit 300 obtains the distance between the position indicated by the position information of the target user 20 received in step s12 and the position indicated by the position information of the specific person acquired in step s44. And the control part 300 determines with the position of the target user 20 and the position of a specific person being near, when the calculated|required distance is less than a threshold value. On the other hand, when the calculated distance is equal to or greater than the threshold, the control unit 300 determines that the position of the target user 20 and the position of the specific person are not close.

If the determination in step s45 is NO, the user identification process ends. In this case, it is determined as NO in step s14. On the other hand, if the determination in step s45 is YES, in step s46 the control unit 300 determines that the specific person is the target user 20, and ends the user identification process. When step s46 is executed, it is determined in step s14 of FIG. 8 that the target user 20 has been specified in the camera image.

If NO is determined in step s43, that is, if a plurality of persons are identified in the person identification process, step s51 is executed as shown in FIG. In step s51, the control section 300 acquires the position information of each of the plurality of specific persons based on the camera image in which the plurality of specific persons are captured, in the same manner as in step s44 described above.

Next, in step s52, the control section 300 determines whether or not there is a specific person whose position is close to the position of the target user among the plurality of specific persons. In step s52, the control section 300 determines whether or not the position of the specific person is close to the position of the target user 20 for each of the plurality of specific persons in the same manner as in step s45 described above.

If NO is determined in step s52, the user identification process ends. In this case, in step s14 of FIG. 8, it is determined that the target user 20 could not be specified in the camera image. On the other hand, if YES is determined in step s52, step s53 is executed. In step s53, the control unit 300 determines whether or not there is only one specific person among the plurality of specific persons whose position is close to the position of the target user.

If YES is determined in step s53, step s46 is executed again. In this step s46, it is determined that the target user 20 is a specific person whose position is close to the position of the target user and which is included in the plurality of specific people.

On the other hand, if NO is determined in step s53, step s54 is executed. In step s54, the control unit 300 determines whether or not step s53 has been performed a specified number of times. The specified number of times is set to, for example, 2 or more.

If determined as YES in step s54, the user identification process ends without identifying the target user 20. FIG. On the other hand, if it is determined as NO in step s54, the control unit 300 executes time elapsing processing in step s55. In other words, the control unit 300 waits for the lapse of a certain period of time. After a certain period of time has elapsed, step s56 is executed. In step s<b>56 , the control unit 300 causes the communication unit 310 to transmit third request information requesting the latest position information of the target user 20 to the center device 5 .

In the center device 5 , when the communication section 510 receives the third request information, the control section 500 causes the communication section 510 to transmit a fourth request signal requesting the latest position information to the target electronic device 2 . In the target electronic device 2 , when the communication section 210 receives the fourth request information, the control section 200 acquires the latest position information from the reception section 230 . Then, the control unit 200 causes the communication unit 210 to transmit the latest position information to the center device 5 . In the center device 5 , when the communication section 510 receives the latest position information from the target electronic device 2 , the control section 500 causes the communication section 510 to transmit the received latest position information to the target roadside device 3 .

After step s56, the communication unit 310 of the target roadside device 3 receives the latest location information of the target user 20 from the center device 5 in step s57. After step s57, step s41 is executed again. At step s41, the control unit 300 executes person identification processing for identifying a person in the latest camera image obtained by the camera 320. FIG. After that, the target roadside unit 3 operates similarly. The latest location information of the target user 20 is used in steps s45 and s52 from the second time onward.

As described above, in the user identification process of this example, the control unit 300 acquires the position information of the person captured in the camera image based on the camera image. Then, the control unit 300 determines whether or not the person is the target user 20 based on the comparison result between the position information of the person captured in the camera image and the position information of the target user 20 . This makes it possible to easily identify the target user 20 in the camera image using the position information.

The roadside device 3 may update the correction information generated by the correction information generation process shown in FIG. FIG. 15 is a flowchart showing an example of correction information update processing. The correction information update process shown in FIG. 15 is repeatedly executed, for example, by the roadside device 3 in operation.

As shown in FIG. 15, in the correction information update process, first, step s61 similar to step s11 described above is executed. As a result, the first request information is transmitted from the target roadside device 3 to the center device 5 . In the center device 5 , the same processing as the processing shown in FIG. 9 is executed, and the position information of the target user 20 is transmitted to the target roadside device 3 .

After step s61, the target roadside unit 3 executes steps s62-s64 similar to steps s12-14 described above. If NO is determined in step s64, step s61 is executed again. Thereafter, the target roadside device 3 operates in the same manner. On the other hand, if determined as YES in step s64, the target roadside device 3 executes steps s65 and s66 similar to steps s15 and s16 described above. As a result, a new location information set consisting of the location information obtained in step s62 and the location information obtained in step s65 is added to storage unit 302. FIG. That is, a position information set consisting of the position information of the user 20 transmitted by the electronic device 2 (that is, the first position information) and the position information of the user 20 that is acquired based on the camera image (that is, the second position information) is newly added to the storage unit 302 . Here, it is assumed that M pairs of position information are stored in the storage unit 302 immediately after step s66 is executed.

After step s66, the target roadside device 3 updates the correction information in the storage unit 302 in step s67 using the position information set newly stored in the storage unit 302 in step s66. In step s67, the control unit 300 newly generates gradient information (in other words, the combined surface 930) using the M sets of position information in the storage unit 302, for example, in the same manner as in step s18 described above. Then, the control unit 300 updates the correction information using the newly generated gradient information as new correction information.

It should be noted that in step s18, when the N pairs of position information are adopted as correction information, in step s67, the control unit 300 adds the position information pairs stored in the storage unit 302 in step s66 to the correction information. By doing so, the correction information is updated.

By updating the correction information in this manner, the roadside device 3 can acquire more accurate position information about the target object.

In the above example, the roadside device 3 acquires the location information of the target user 20 from the center device 5 , but may acquire the location information directly from the electronic device 2 without going through the center device 5 . For example, consider a case where the roadside device 3 can directly communicate with the electronic device 2 using the 700 MHz band assigned to the ITS. Hereinafter, communication using the 700 MHz band assigned to ITS may be referred to as 700 MHz band communication.

Since the communication distance of the 700 MHz band communication is not very long, when the target roadside device 3 receives the position information directly from the electronic device 2 via the 700 MHz band communication, it is assumed that the electronic device 2 exists at a position close to the target roadside device 3. I can say In other words, when the target roadside device 3 receives the position information directly from the electronic device 2 through 700 MHz band communication, there is a possibility that the image obtained by the camera 320 of the target roadside device 3 includes the user 20 of the electronic device 2. expensive. Therefore, when the target roadside device 3 receives the position information directly from the electronic device 2 through 700 MHz band communication, it can be said that the user 20 of the electronic device 2 is the target user 20 . Therefore, when the target roadside device 3 directly receives the position information from the electronic device 2 through the 700 MHz band communication, the target roadside device 3 may use the position information as the position information of the target user 20 and execute the processing from step s13 onward in FIG. good. Further, when the target roadside device 3 directly receives the position information from the electronic device 2 through the 700 MHz band communication, the target roadside device 3 may use the position information as the position information of the target user 20 and execute the processing from step s63 onward in FIG. good. In such a case, the center device 5 becomes unnecessary.

Although the communication system 1 has been described in detail as above, the above description is illustrative in all aspects, and the present disclosure is not limited thereto. Moreover, the various modifications described above can be applied in combination as long as they do not contradict each other. And it is understood that numerous variations not illustrated can be envisioned without departing from the scope of this disclosure.

3 roadside device 300 control unit 320 camera

Claims (6)

a camera for obtaining a first image;
First position information indicating the absolute position of a vehicle or pedestrian to be detected, which is captured in the first image obtained based on the installation state of the camera and the first image, is converted to correction information generated in advance. and a processing unit that corrects based on
A second position indicating an absolute position of a pedestrian captured in a second image obtained by the camera before the first image, which is obtained based on the installation situation and the second image. Acquiring a plurality of sets in which information is associated with third position information indicating an absolute position transmitted by a device possessed by the pedestrian, and generating the correction information based on the obtained plurality of sets; Device. The imaging device according to claim 1,
The processing unit generates, as the correction information, information representing a combined plane having a gradient with respect to a reference plane including the positions indicated by the plurality of second position information,
The combined plane is composed of a plurality of planes generated based on a plurality of the third position information,
The imaging device, wherein the processing unit corrects the first position information based on the position of the intersection of the combined plane and a line extending in a predetermined direction from the position indicated by the first position information on the reference plane. The imaging device according to any one of claims 1 and 2,
When the detection target is a pedestrian, the processing unit determines that the pedestrian exists on the roadway based on the first position information corrected based on the correction information, and determines that the pedestrian exists on the roadway. An imaging device that performs a process of notifying the imaging device of the fact. The imaging device according to any one of claims 1 to 3,
When the detection target is a vehicle, the processing unit performs the detection based on the first position information corrected based on the correction information and the corrected first position information of the vehicle acquired earlier. An imaging device that, when determining that the vehicle is stopped on the roadway, performs processing for causing the imaging device to notify that the vehicle is stopped on the roadway. A control program for controlling the operation of an imaging device equipped with a camera for obtaining a first image,
In the imaging device,
First position information indicating the absolute position of a vehicle or pedestrian to be detected, which is captured in the first image obtained based on the installation state of the camera and the first image, is converted to correction information generated in advance. correcting based on
Second positional information indicating an absolute position of a pedestrian captured in a second image obtained by the camera before the first image, obtained based on the installation situation and the second image; and the walking. obtaining a plurality of sets associated with the third position information indicating the absolute position transmitted by the device possessed by the person, and generating the correction information based on the obtained plurality of sets. control program. A method of operating an imaging device comprising a camera for obtaining a first image, comprising:
First position information indicating the absolute position of a vehicle or pedestrian to be detected, which is captured in the first image obtained based on the installation state of the camera and the first image, is converted to correction information generated in advance. correcting based on
Second position information indicating an absolute position of a pedestrian captured in a second image obtained by the camera before the first image, the second position information obtained based on the installation situation and the second image; and the walking. obtaining a plurality of sets associated with third position information indicating an absolute position transmitted by a device owned by a person, and generating the correction information based on the obtained plurality of sets. How it works.

Images