JP2023025053A - Determination device, determination method, and determination program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To control plural devices in a coordinated manner based on determination of the identity of images captured by the devices.

SOLUTION: A determination device comprises: a first acquisition unit 101 which acquires a first image captured by a first device together with a capturing time; a second acquisition unit 102 which acquires a second image captured by a second device together with a capturing time; a determination unit 103 which determines the identity of the capturing positions of the first image and the second image, on the basis of the respective captured images and capturing times of the first image and the second image; and an output unit 105 which outputs a determination result to the first device or the second device.

SELECTED DRAWING: Figure 1

COPYRIGHT: (C)2023,JPO&INPIT

Description

The present invention relates to a determination device, a determination method, and a determination program for determining whether or not a plurality of devices can cooperate based on images captured by the respective devices. However, utilization of the present invention is not limited to the determination device, determination method, and determination program.

Conventionally, a technology has been proposed that causes a plurality of devices to operate in cooperation with each other. For example, a fingerprint sensor is provided in a pointing device common to the first device and the second device, personal identification information detected by the fingerprint sensor of one device is notified to the other device, and the other device notifies the cooperative operation, A technology has been disclosed in which a predetermined cooperative operation such as user authentication is performed upon matching notification of personal identification information from one device (see, for example, Patent Document 1 below).

JP 2008-181319 A

However, in the above-described conventional technology, a complicated connection procedure is required for cooperative operation of a plurality of devices for authentication, and user operations are time-consuming.

In order to solve the above-described problems and achieve the object, the determination device according to the first aspect of the invention includes first acquisition means for acquiring a first image taken by a first device, and a second acquisition unit that acquires a second image; a determination unit that determines identity of photographing positions of the first image and the second image based on the photographed images of the first image and the second image; and output means for outputting to the first device or the second device that the first device and the second device can cooperate with each other according to the determination result of the determination means. .

A determination method according to a sixth aspect of the present invention is a determination method performed by a determination device, in which a first acquisition step of acquiring a first image captured by a first device and a second image captured by a second device are obtained. a second acquisition step of acquiring an image; a determination step of determining identity of photographing positions of the first image and the second image based on the photographed images of the first image and the second image; and an outputting step of outputting to the first device or the second device that the first device and the second device can cooperate with each other according to the determination result of the step.

A determination program according to a seventh aspect of the invention causes a computer to execute the determination method according to the sixth aspect.

1 is a block diagram illustrating an example of a functional configuration of a determination device according to an embodiment; FIG. 2 is a flowchart illustrating an example of a processing procedure of the determination device according to the embodiment; FIG. FIG. 3 is a diagram illustrating a configuration example of a plurality of devices according to the first embodiment; FIG. 4 is a block diagram showing an example of the hardware configuration of the navigation device according to the first embodiment. 5 is a flowchart illustrating an example of determination processing executed by the first device according to the first embodiment; FIG. FIG. 6 is a flowchart illustrating an example of a same vehicle determination process using an image executed by the first device according to the first embodiment. FIG. 7 is a diagram for explaining detection parameters detected in image processing according to the first embodiment. FIG. 8 is a diagram showing an example of a notification screen of the same vehicle determination process according to the first embodiment. FIG. 9 is a diagram illustrating an example of cooperative control between the first device and the second device according to the first embodiment. (Part 1) FIG. 10 is a diagram illustrating an example of cooperative control between the first device and the second device according to the first embodiment. (Part 2) FIG. 11 is a diagram illustrating an example of cooperative control between the first device and the second device according to the first embodiment. (Part 3) FIG. 12 is a diagram illustrating an example of cooperative control between the first device and the second device according to the first embodiment. (Part 4) FIG. 13 is a diagram illustrating a configuration example of a plurality of devices according to the second embodiment; FIG. 14 is a flowchart illustrating an example of determination processing executed by the third device according to the second embodiment;

(Embodiment)
Exemplary embodiments of a determination device, a determination method, and a determination program according to the present invention will be described in detail below with reference to the accompanying drawings. According to the present invention, images captured by a plurality of devices are subjected to image processing or the like to determine whether the shooting positions of the plurality of devices are the same, and the plurality of devices determined to be the same are linked. For example, without manually setting a plurality of devices mounted on a certain vehicle, it is possible to automatically determine whether or not a plurality of devices can be cooperatively controlled. Since these devices have different OSs, for example, they are devices that cannot be cooperatively controlled simply by connecting them to a network.

1 is a block diagram illustrating an example of a functional configuration of a determination device according to an embodiment; FIG. The determination device 100 according to the embodiment includes a first acquisition unit 101 , a second acquisition unit 102 , a determination unit 103 , a display control unit 104 and an output unit 105 . In embodiments, a single information appliance, such as a server, has each of these functions.

A first acquisition unit 101 acquires a first image captured by a first device from the first device. A second acquisition unit 102 acquires a second image captured by the second device from the second device. Also, the first acquisition unit 101 and the second acquisition unit 102 may acquire the shooting time together with the image. The first image and the second image are not limited to still images, and may be moving images (set of still images by time).

Based on the photographed images of the first image acquired by the first acquisition unit 101 and the second image acquired by the second acquisition unit 102, the determination unit 103 determines the identity of the shooting positions of the first image and the second image. judge. This determination may include the shooting time.

The output unit 105 notifies the first device and the second device whether or not they can cooperate with each other according to the determination result of the determination unit 103 .

The display control unit 104 performs display control for causing the second device to display a part of the image that can be displayed on the first device based on the determination result of the determination unit 103 .

The above configuration will be described with an example in which the first device and the second device are mounted on the same vehicle, for example. It is assumed that the first device and the second device have a photographing unit (camera) directed in a predetermined direction, the camera photographs a still image, and outputs the first image and the second image to the determination device 100 . In this case, the determination unit 103 determines whether the photographing positions of the first image and the second image are the same.

In this determination of identity of shooting positions, it is determined whether or not the first image and the second image were shot from the same position by performing image processing on the detected object (subject) indicated by the images. Here, determination is not made based only on strict matching between the first image and the second image.

In this example, it is determined whether the images were taken from the same vehicle (corresponding to whether the first device and the second device are mounted on the same vehicle). Therefore, in the determination of the sameness of shooting positions, if the first image and the second image both contain the same detection object (subject), the difference in the installation positions of the first device and the second device and the shooting It is determined whether or not the images were taken within the range of the same vehicle while allowing for a difference in angle of view.

At the time of determination by this image processing, it is possible to further determine whether the photographing times of the first image and the second image are the same. For example, if the shooting times of the first image and the second image were both taken within a predetermined period of time, a weight is added to the determination result that the images were taken from the same position to determine the identity of the shooting positions. improve the credibility of

In addition, it is possible to cope with the case where the shooting direction of the first device and the shooting direction of the second device have different angles. For example, the determination unit 103 performs image processing on the content (subject) of the image captured in both the first image and the second image in consideration of different shooting angles in advance, and further includes the determination of the shooting time. It can be determined whether or not the first image and the second image are taken from the same position (same vehicle).

Furthermore, it is possible to cope with the case where the shooting direction of the first device and the shooting direction of the second device are opposite directions. For example, when the first device captures the front of the vehicle, the second device captures the rear of the vehicle, and the vehicle moves forward, the first device captures the subject in front of the vehicle in the direction of travel. After the corresponding predetermined time elapses, the second device photographs the same subject. Even in this case, the determination unit 103 determines the difference between the first image and the second image, taking into consideration in advance that the shooting directions are opposite and that the second device will shoot the same subject after a predetermined time has passed since the shooting by the first device. In either case, it is possible to determine whether or not the first image and the second image were taken from the same position (same vehicle) by image processing the content (object) of the image taken.

In this case, the first device may photograph one side of the subject, and the second device may photograph the other side of the same subject, and the image processing is performed in consideration of the difference between the planes. For example, for a road sign, the first image will be taken of the sign face and the second image will be taken of the back face of the sign. In this case, the first image is image-recognized as a sign, including the sign surface and shape, and the second image does not include the sign surface and the shape of the image corresponding to the sign after a predetermined period of time matches. If it is possible to recognize that it is (sign), the determination unit 103 determines that the first image and the second image were shot from the same vehicle.

For example, the determination device 100 is communicatively connected to the first device and the second device via a network, and communicates information (eg, the first image, the second image, determination results, etc.) with each other via the network. For example, there are various wired and wireless connection forms such as using an in-vehicle communication network and wireless LAN.

Here, the determination device 100 links the first device and the second device without the user performing a setting operation. In order to control this cooperation, when the above-mentioned captured images are acquired wirelessly, for example, not only the image of the equipment installed in the own vehicle, but also the image captured by the equipment of another vehicle running adjacently is acquired. can be considered. To deal with this, for example, the determination device 100 (determination unit 103) can determine whether or not the first device and the second device are provided in the same vehicle by matching the SSID of the wireless LAN. Also, if the driving lane and inter-vehicle distance included in the first image and the second image are within a predetermined range, it can be determined that they are the same vehicle. Furthermore, if the activation times (ACC ON of the vehicle) of the first device and the second device match, it is possible to determine whether the first device and the second device are provided in the same vehicle.

Then, if the result of determination by the determination unit 103 is that the first device and the second device can cooperate, the display control unit 104 of the determination device 100 displays a part of the image that can be displayed on the first device on the second device. display control. For example, a part of the image that can be displayed on the first device is displayed on the second device. When the first device and the second device each have a display unit, the display content to be displayed by the first device and the display content to be displayed by the second device can be different. You can select the display contents according to the

For example, a first device provided on the front seat side of the vehicle can display driving information, and a second device provided on the back seat side can display entertainment information. Further, when the first device is a navigation device and a mirror-type terminal is provided as the second device in the rearview mirror, the navigation device displays a map, and the mirror-type terminal displays guidance (such as displaying the direction of travel at the next intersection). )It can be performed.

Then, the first device and the second device can also recognize whether or not they can cooperate with each other according to the notification of the determination result output by the output unit 105 of the determination device 100 . For example, after receiving the notification that the first device and the second device are capable of cooperating with each other, the determination device 100 directly communicates with the first device and the second device without mediating the exchange of information between the first device and the second device. Linked control (for example, display control for causing the second device to display part of the image that can be displayed on the first device) can also be performed between the first device and the second device.

In the above description, the determination device 100, the first device, and the second device are described as separate devices. can also In this case, the first device has each of the above functions (see FIG. 1), and it is determined whether or not the second device is mounted in the same vehicle as the first device (own device), and cooperation with the second device is performed. You can control it.

2 is a flowchart illustrating an example of a processing procedure of the determination device according to the embodiment; FIG. 3 shows an example of determination and cooperation control executed by the determination device 100. FIG.

First, in the determination device 100, the first acquisition unit 101 acquires a first image from a first device (step S201), and the second acquisition unit 102 acquires a second image from a second device (step S202). Next, in the determination device 100, the determination unit 103 determines whether the shooting positions of the first image and the second image are the same (step S203). In this determination, as described above, it is determined whether the first image and the second image were shot from the same vehicle. The determination may include the shooting time in addition to the captured image.

As a result of the determination in step S203, if it is determined that they are the same (the shooting positions are the same) (step S203: Yes), the output unit 105 of the determination device 100 causes the first device and the second device to notifies that cooperation is possible (step S204). Then, the display control unit 104 of the determination device cooperatively controls the display of the first device and the second device (step S205). For example, as described above, this cooperative control causes the second device to display part of the video that can be displayed on the first device.

On the other hand, if the result of determination in step S203 is that they are not the same (no identity in the shooting positions) (step S203: No), the output unit 105 of the determination device 100 terminates the process. Alternatively, the output unit 105 of the determination device 100 may notify the first device and the second device that they cannot cooperate with each other.

FIG. 2 shows an example of processing in which the determination device 100 determines the identity of the shooting positions of the first device and the second device and performs cooperative control. A determination device 100 in the device determines the identity of the photographing position (same vehicle) and cooperatively controls the second device.

In the above description, the devices to be linked are the first device and the second device. Based on the 1st to n-th images captured by the devices, it is possible to determine the identity of the photographing positions and to cooperatively control the 1st to n-th devices.

According to the above-described embodiments, it is possible to determine whether the photographed positions of a plurality of devices are the same using images captured by a plurality of devices, and to perform coordinated control between the devices determined to be identical. In addition, users can easily link devices without performing complicated connection procedures.

FIG. 3 is a diagram illustrating a configuration example of a plurality of devices according to the first embodiment; In the first embodiment, a user's vehicle is equipped with a navigation device 300 as a first device and an information device 302 as a second device. will be used for explanation.

The first device (information device 1: navigation device) 300 and the second device (information device 2) 302 respectively have cameras 311 and 312 for capturing images and communication units 321 and 322 for mutual communication. A second device (information device 2 ) 302 has a function of transmitting captured images to the first device (navigation device) 300 and a function of receiving cooperative control from the first device (navigation device) 300 .

(Hardware configuration of the first device)
FIG. 4 is a block diagram showing an example of the hardware configuration of the navigation device according to the first embodiment. In FIG. 4, the navigation device 300 as the first device includes a CPU 401, a ROM 402, a RAM 403, a magnetic disk drive 404, a magnetic disk 405, an optical disk drive 406, an optical disk 407, an audio I/F (interface) 408, a microphone 409, and a speaker 410. , an input device 411 , a video I/F 412 , a display 413 , a communication I/F 414 , a GPS unit 415 , various sensors 416 and a camera 417 . Each component 401 to 417 is connected by a bus 420 respectively.

The CPU 401 controls the navigation device 300 as a whole. A ROM 402 records a boot program and a judgment program. A RAM 403 is used as a work area for the CPU 401 . That is, the CPU 401 controls the entire navigation device 300 by executing various programs recorded in the ROM 402 while using the RAM 403 as a work area.

The magnetic disk drive 404 controls reading/writing of data to/from the magnetic disk 405 under the control of the CPU 401 . The magnetic disk 405 records data written under the control of the magnetic disk drive 404 . As the magnetic disk 405, for example, an HD (hard disk) or an FD (flexible disk) can be used.

Also, the optical disk drive 406 controls reading/writing of data to/from the optical disk 407 under the control of the CPU 401 . The optical disc 407 is a detachable recording medium from which data is read under the control of the optical disc drive 406 . A writable recording medium can also be used for the optical disc 407 . As a removable recording medium, in addition to the optical disc 407, an MO, a memory card, or the like can be used.

Examples of information recorded on magnetic disk 405 and optical disk 407 include map data, vehicle information, road information, and travel history. Map data is used for route search in car navigation systems, and includes background data representing features such as buildings, rivers, ground surfaces, and energy supply facilities, and road geometry representing road shapes with links and nodes. It is vector data including data and so on.

Audio I/F 408 is connected to microphone 409 for audio input and speaker 410 for audio output. Sound received by the microphone 409 is A/D converted within the sound I/F 408 . The microphone 409 is installed, for example, on the dashboard of the vehicle, and the number of microphones 409 may be singular or plural. The speaker 410 outputs audio obtained by D/A converting a predetermined audio signal in the audio I/F 408 .

Examples of the input device 411 include a remote controller, a keyboard, a touch panel, and the like having a plurality of keys for inputting characters, numerical values, various instructions, and the like. The input device 411 may be implemented in any one form of a remote controller, keyboard, or touch panel, but may be implemented in a plurality of forms.

Video I/F 412 is connected to display 413 . Specifically, the video I/F 412 includes, for example, a graphic controller that controls the entire display 413, a buffer memory such as a VRAM (Video RAM) that temporarily records image information that can be displayed immediately, and an output from the graphic controller. It is composed of a control IC or the like that controls the display 413 based on the image data received.

The display 413 displays icons, cursors, menus, windows, or various data such as characters and images. As the display 413, for example, a TFT liquid crystal display, an organic EL display, or the like can be used.

Camera 417 captures an image including the road outside the vehicle. The image may be either a still image or a moving image. For example, the outside of the vehicle is captured by the camera 417, the captured image is image-analyzed by the CPU 401, or the recording medium such as the magnetic disk 405 or the optical disk 407 is transmitted via the image I/F 412. output to

Communication I/F 414 is wirelessly connected to a network and functions as an interface for navigation device 300 and CPU 401 . Communication networks that function as networks include in-vehicle communication networks such as CAN and LIN (Local Interconnect Network), public telephone networks, mobile phone networks, DSRC (Dedicated Short Range Communication), LAN, and WAN. The communication I/F 414 is, for example, a public line connection module, an ETC (non-stop automatic toll payment system, registered trademark) unit, an FM tuner, a VICS (Vehicle Information and Communication System: registered trademark)/beacon receiver, and the like.

The GPS unit 415 receives radio waves from GPS satellites and outputs information indicating the current position of the vehicle. The output information of the GPS unit 415 is used together with the output values of various sensors 416, which will be described later, when the CPU 401 calculates the current position of the vehicle. The information indicating the current position is, for example, information specifying one point on the map data such as latitude/longitude and altitude.

Various sensors 416 output information for determining the position and behavior of the vehicle, such as a vehicle speed sensor, an acceleration sensor, an angular velocity sensor, and an inclination sensor. The output values of the various sensors 416 are used by the CPU 401 to calculate the current position of the vehicle and the amount of change in speed and direction.

Using the programs and data recorded in the ROM 402, RAM 403, magnetic disk 405, optical disk 407, etc. shown in FIG. The functions of the determination unit 103 and the display control unit 104 of the determination device 100 shown in 1 are realized. That is, in the first embodiment, since the determination device 100 is incorporated in the navigation device 300 , the image captured by the camera 417 of the navigation device 300 is acquired by the CPU 401 of the navigation device 300 . Further, the functions of the first acquisition unit 101 and the output unit 105 in FIG. 1 can be realized using the communication I/F 414 in FIG. 4, and the display 413 in FIG. output) can be realized.

(Hardware configuration of the second device)
The hardware configuration of the second device (information device 2) 302 of the first embodiment includes CPU 401 to RAM 403, video I/F 412, camera 417, display 413, and communication I/F 414 among the units shown in FIG. can be configured with

In this second device 302 as well, the CPU 401 executes a predetermined program using programs and data recorded in the ROM 402, RAM 403, etc., shown in FIG. device) 300 and receive cooperative control by the first device (navigation device) 300, for example, the display image received from the first device (navigation device) 300 is displayed on the display 413.

(Example of determination processing of the determination device of the first embodiment)
Next, the first device (navigation device) 300 having the function of the determination device 100 determines the identity of the photographed positions, that is, the same vehicle, based on the photographed image of its own device and the photographed image of the second device (information device 2) 302. A control example will be described in which it is determined whether or not the image is captured in (whether the first device and the second device are provided in the same vehicle), and the first device and the second device are linked.

5 is a flowchart illustrating an example of determination processing executed by the first device according to the first embodiment; FIG. FIG. 5 also shows operation processing of the second device.

First, when performing coordinated control of other information equipment mounted in the same vehicle, the first equipment 300 starts searching for the second equipment (information equipment 2) 302 based on a predetermined operation (step S501: Yes). When the first device 300 does not perform cooperative control of other information devices, the first device 300 does not start searching for the second device (information device 2) 302 (step S501: No), and ends the following processing. .

Next, the first device 300 captures an image with its own camera 311 to obtain a first image (step S502). The first device 300 captures an image with the camera 311 at a predetermined timing (regularly or irregularly). At this time, the first device 300 acquires the photographing time as the photographing parameter of the first image and associates it with the photographed image (step S503). In this step S503, the state of the vehicle (for example, speed, traveling direction, acceleration, etc. from various sensors 416) and the shooting state (angle of view, exposure, etc.) may be acquired as other shooting parameters.

Next, the first device 300 acquires the second image captured by the second device 302 by receiving it from the second device 302 (step S504). The second device 302 captures an image with the camera 312 at a predetermined timing (regularly or irregularly). Here, the second image acquired by the first device 300 may selectively receive an image that matches the shooting time of the first image. You may request to send an image.

Also, the first image and the second image are not limited to still images and may be moving images. Using the shooting time of the first image as a reference, the shooting time of the second image overlaps with the shooting time of the first image. It may be acquired. The first device 300 can determine the identity of the shooting positions by using the images (still images) captured at the same time among the first image and the second image, which are moving images. The first device 300 saves the acquired first image and second image in a storage unit (for example, the magnetic disk 405 in FIG. 4).

Next, the first device 300 reads out the first image and the second image from the storage unit, and determines whether the first image and the second image are images captured by the same vehicle (step S505). This process corresponds to the determination of the identity of the photographing positions, and is determined by image processing.

Although an example of processing for determining the identity of photographing positions will be described later, the first image and the second image photographed at the same time are read out and compared, and the first image and the second image are determined based on whether or not there is a matching subject. are images taken by the same vehicle. At this time, strictly speaking, the first image and the second image are compared on the assumption that there is no image that completely matches the first image and the second image. The photographing time of the image and the photographing time of the second image are made different, or the images are compared after performing correction corresponding to the difference in the angle of view between the first image and the second image.

As a result of the determination processing in step S505, if it is determined that the first image and the second image are images captured by the same vehicle (step S505: Yes), the first device 300 cooperatively controls the second device 302. (step S506), and thereafter, the second device 302 is notified of the determination result and cooperative control is performed (an example of cooperative control will be described later). As a result of the determination processing in step S505, if it is not determined that the first image and the second image are images shot by the same vehicle (step S505: No), the process returns to step S501.

On the other hand, when the second device (information device 2) 302 captures an image (second image) (step S511), it acquires the capture time of the second image and associates it with the captured image (step S512). In this step S512, other information related to photography that can be obtained by the second device may be obtained.

Then, the second device 302 transmits the photographed second image together with the photographing time and other information (imaging parameters) related to photographing to the first device (step S513).

After that, the second device 302 waits for notification of the determination result of the identity of the shooting positions from the first device 300 . When the first device 300 determines that the photographing positions are identical, the second device 302 is notified by the first device 300 that the second device 302 is an image photographed by the same vehicle as the first device 300 . be. As this notification, it may be notified that the second device 302 is mounted in the same vehicle as the first device 300 . After that, the second device 302 receives cooperative control from the first device 301 (step S514).

(Example of image processing)
FIG. 6 is a flowchart illustrating an example of a same vehicle determination process using an image executed by the first device according to the first embodiment. An example of processing executed by the determination device 100 of the first device 300 is shown, and details of the processing (step S505 in FIG. 5) performed on each of the first image and the second image are described.

First, the first device 300 acquires shooting parameters including shooting times of the first image and the second image (step S601). Next, the first device 300 determines whether the first image and the second image are captured at the same time (step S602). Here, for the sake of convenience, it is assumed that the first image and the second image are taken at the same time. If there is a first image and a second image taken at the same time (step S602: Yes), the processing from step S603 onwards is executed, and if there is no first image and second image taken at the same time, (Step S602: No), the process ends.

In step S603, the image information of the first image and the second image are respectively detected by image processing (step S603). In this process, for example, image edge detection is performed (step S611), lane area detection is performed (step S612), object detection is performed (step S613), and each detection parameter is saved (step S614). ).

Thereafter, the first device determines whether each detection parameter detected in step S603 is the same for the first image and the second image (step S604). As a result of the determination in step S604, if the detection parameters of the first image and the second image are the same (step S604: Yes), it is determined that the first image and the second image were captured by the same vehicle (step S605). ). On the other hand, if the detection parameters of the first image and the second image are not the same as the determination result of step S604 (step S604: No), it is determined that the first image and the second image are not images shot by the same vehicle. Then, the processing in FIG. 6 is terminated (return to step S501 in FIG. 5).

FIG. 7 is a diagram for explaining detection parameters detected in image processing according to the first embodiment. An image 700 detected for each of the first image and the second image in the image processing performed by the first device is shown, and each detection parameter is exemplified. Edge detection (step S611) detects edges on the image, and after this edge detection, lane areas (detection lanes 1 to 4 in the figure) are detected (step S612).

At this time, the area divided by the white lines between the lanes is recognized as the lane area (detected lanes 1 to 4 in FIG. 7). Also, the lane closest to the center in the horizontal direction of the image (detected lane 2 in FIG. 7) is recognized as the lane in which the vehicle is traveling.

As for object detection (step S613), the vehicle 701 is detected on the lane. For example, by machine-learning the shape of an object (vehicle) using the Adaboost algorithm, the vehicle 701 can be detected by vehicle type. Similarly, a road sign 702 located on the roadside (the left end of detection lane 1 in FIG. 7) can be detected as a shape-specific sign.

(Example of determination processing for each installation position of the first camera and the second camera)
For example, when both the first device 300 and the second device 302 capture an image of the front of the vehicle, the positional relationship between the lane in which the vehicle is traveling and the detected vehicle obtained by image processing is determined by the two devices 300 and 302. , it is determined that the first image and the second image are images captured by the same vehicle.

Further, when the first device 300 captures an image in front of the vehicle and the second device 302 captures an image behind the vehicle, and the vehicle is traveling forward, the first image captured by the first device 300 and the In the image captured by the second device 302, the positional relationship of the subject (a stationary object such as a road sign or a vehicle parked on the road shoulder) obtained by image processing is reversed between the first image and the second image. Judge by

Furthermore, for example, in the case of a road sign, the road sign has a front and a back. The determination can be made later based on photographing the reverse side of the same road sign 702 located at the right end of the road with the second device 302 .

(Notification example after same vehicle determination)
FIG. 8 is a diagram showing an example of a notification screen of the same vehicle determination process according to the first embodiment. 6 shows an example of a screen on which the first device 300 notifies the second device 302 in step S506 of FIG. For example, when the first device 300 determines that the first image and the second image are images captured by the same vehicle as a result of performing the same vehicle determination process described above, the first device 300 causes the second device 302 to In response, an authentication screen 800 shown in FIG. 8 is transmitted.

In this authentication screen 800, the photographed image 801 photographed by the second device 302 and used for the determination of the image photographed by the same vehicle as described above is transmitted, and "A device that can be synchronized with the vehicle being driven is detected. ," is displayed. For example, it is a new terminal when this second device 302 is newly installed in a vehicle.

In this way, the first device 300 having the function of the determination device 100 can be used without any user operation every time a new terminal (second device 302) is newly attached to the vehicle. The user can be notified by notifying the device 302) that the device is installed in the same vehicle. If the user has installed the second terminal 302 in the same vehicle, the user can synchronize (coordinate) the second device 302 with the first device 300 by operating the “synchronize” selection button 802 on the authentication screen 800 . ). Further, by operating the “do not synchronize” selection button 803, the second device 302 can not be synchronized (coordinated) with the first device 300. FIG.

As described above, the first device 300 can synchronize (coordinate) the first device 300 and the second device 302 after confirming synchronization (cooperation) with the user as shown in FIG. Not limited to this, the first device 300 automatically synchronizes (coordinates) the first device 300 and the second device 302 without sending a synchronization (coordination) confirmation screen to the user as shown in FIG. may

(Example of cooperative control between the first device and the second device)
Next, when the first device 300 and the second device 302 described above are registered as images taken by the same vehicle (step S506 in FIG. 5), the first device 300 and the second device 302 are registered. An example of synchronization (cooperation) control with the device 302 will be described.

9 to 12 are diagrams showing examples of cooperative control of the first device and the second device according to the first embodiment. 9 to 12, the first device 300 cooperates the display of the second device 302 based on the determination result that the first device (navigation device) 300 and the second device 302 are provided in the same vehicle. An example of control is described. Assume that the second device 302 is a mirror-type terminal having a display function provided in the rearview mirror.

In the example shown in FIG. 9, the first device (navigation device) 300 displays a map screen 901 including the route as a result of the route search. Also, the first device (navigation device) 300 controls the display of the second device 302 and displays information 902 on the route (for example, the guidance direction of the next intersection) while the vehicle is traveling.

In the example shown in FIG. 10, the first device (navigation device) 300 displays a map screen 901 including the route as a result of the route search. Also, the first device (navigation device) 300 controls the display of the second device 302 and displays the music list of the content being reproduced on the first device (navigation device) 300 and the video 1002 .

In the example shown in FIG. 11, the first device (navigation device) 300 displays a song list and video 1101 of the content being reproduced. In addition, the first device (navigation device) 300 controls the display of the second device 302, and based on the route searched by the first device (navigation device) 300, information on the route (for example, , the guidance direction of the next intersection) 902 is displayed.

In the example shown in FIG. 12, the first device (navigation device) 300 displays an image 1201 captured by the camera (front camera) 311a connected to the first device. The first device (navigation device) 300 also controls the display of the second device 302 and displays an image 1202 captured by the camera (rear camera) 311b connected to the first device (navigation device) 300 . Note that the second device 302 itself has a front camera 312 and can display the front camera 312 on the second device 302 .

As shown in FIGS. 9 to 12, after determining that the first device (navigation device) 300 and the second device (mirror type terminal) 302 are provided in the same vehicle, the first device 300 , the second device 302 can be cooperatively controlled (synchronized control with the progress of the vehicle, etc.). As in the above example, by linking screen displays, information that cannot be displayed on one device can be displayed on another device. Furthermore, it becomes possible to display necessary information on one device and on the other device. In the above example, the displays of the first device 300 and the second device 302 are cooperatively controlled. devices can be linked and controlled.

According to the first embodiment described above, by providing the function of the determination device 100 in the first device (navigation device) 300, when the first device (navigation device) 300 detects the second device 302, the second device 302 is detected. It is determined whether the device 302 is provided in the same vehicle as the own device. At this time, the first device 300 determines by image processing whether the first image and the second image are images shot from the same vehicle. By acquiring the second image captured by the second device 302, the first device 300 compares the image with the first image of its own device to determine whether the second device 302 is provided in the same vehicle. I can judge.

As a result, when the second device 302 is installed, the second device 302 can be cooperatively controlled without performing complicated user settings to the effect that the second device 302 has been installed. Note that the second device 302 is not limited to a single device and can be a plurality of devices, and the first device 300 can cooperatively control the plurality of second devices 302 .

FIG. 13 is a diagram illustrating a configuration example of a plurality of devices according to the second embodiment; The second embodiment is an example in which the device configuration is changed from that of the first embodiment. As shown in FIG. 13, in the second embodiment, a user's vehicle is equipped with a navigation device 300 as a first device (information device 1) and a second device (information device 2, for example, a mirror type terminal similar to the first embodiment) 302. , a server 1301 as a third device (information device 3) is mounted, and the server 1301 as the third device has the functions of the determination device 100 described above.

A first device (navigation device) 300 and a second device (information device 2) 302 respectively have cameras 311 and 312 for capturing images and communication units 321 and 322 for mutual communication. A first device (navigation device) 300 and a second device (information device 2) 302 each have a function of transmitting captured images to a third device (server) 1301 and cooperative control by the third device (server) 1301. has the function of receiving

(Hardware configuration of the second embodiment)
The hardware configuration of the first device (navigation device) 300 of Example 2 is the same as in FIG. The hardware configuration of the second device (information device 2) 302 includes CPU 401 to RAM 403, video I/F 412, camera 417, display 413, and communication I/F 414 among the units shown in FIG. can be configured Further, the hardware configuration of the third device (server) 1301 can be configured by including the CPU 401 to RAM 403 and the communication I/F 414 among the units shown in FIG.

The third device (server) 1301 uses programs and data recorded in the ROM 402, the RAM 403, the magnetic disk 405, the optical disk 407, etc. shown in FIG. The function of the determination device 100 can be realized. Then, the third device (server) 1301 acquires the captured images captured by the cameras 311 and 312 of the first device (navigation device) 300 and the second device (information device 2) 302 via the communication unit 1302, and , determines whether the first device (navigation device) 300 and the second device (information device 2) 302 are attached to the same vehicle by performing image processing on the captured image, and transmits the first device (navigation device) 300 and a second device (information device 2) 302 are cooperatively controlled.

The third device (server) 1301 sends the first device (navigation device) 300 and the second device (information device 2) 302 to the same vehicle. 302 may be controlled. In this case, the first device 300 and the second device 302 receive the notification that they can cooperate with each other, and without going through the third device (server) 1301, the first device 300 and the second device 302 directly cooperate. control.

(Example of determination processing of the determination device of the second embodiment)
Next, a third device (server) 1301 having the function of the determination device 100 compares the first image captured by the first device (navigation device) 300 and the second image captured by the second device (information device 2) 302. Based on this, it is determined whether or not the photographed positions are the same, that is, whether the images were photographed in the same vehicle (whether the first device and the second device are provided in the same vehicle), and the first device and the second device are determined. An example of control to be linked will be described.

FIG. 14 is a flowchart illustrating an example of determination processing executed by the third device according to the second embodiment; FIG. 14 also shows operation processing of the first device and the second device.

The first device 300 and the second device 302 capture images with the cameras 311 and 312 at predetermined timing (irregularly or regularly), respectively (step S1410). Taking the first device 300 as an example, the first device 300 captures an image with the camera 311 to obtain a first image (step S1410). At this time, the first device 300 acquires the photographing time as the photographing parameter of the first image and associates it with the photographed image (step S1411).

In this step S1411, the shooting state (for example, angle of view, exposure) may be acquired as another shooting parameter. In this step S1411, other information related to photography that can be obtained by the first device may be obtained. The first device 300 then transmits the data of the first image together with the imaging parameters to the third device 1301 (step S1412). As with the first device 300 , the second device 302 assigns shooting parameters to the second device 302 and transmits the second image to the third device 1301 each time the second image is shot.

Each time the first image and the second image are transmitted from the first device (navigation device) 300 and the second device (information device 2) 302 mounted on the same vehicle, the third device 1301 receives the first image and the second image. 2 images (data) are received (step S1401) and stored in a storage unit (for example, the magnetic disk 405 in FIG. 4) (step S1402). Here, the third device 1301 receives images transmitted from, for example, the first device 300 and the second device 302 newly attached to the same vehicle.

After that, the third device 1301 performs the following processing to determine whether the photographing positions of the acquired first image and second image are the same. First, the third device 1301 acquires the first image and the second image captured at the same time among the acquired first image and second image (step S1403).

By the way, the first image and the second image are not limited to still images, and may be moving images. The identity of the photographing position can be determined using the image (still image) photographed at the same time among the first image and the second image.

Next, the third device 1301 determines whether the first image and the second image are images captured by the same vehicle (step S1404). This process corresponds to the determination of the identity of the photographing positions, and is determined by image processing.

As described with reference to FIG. 6, the example of processing for determining the identity of photographing positions is to compare a first image with a second image photographed at the same time, and determine whether or not there is a matching subject. It is determined whether the second image is an image captured by the same vehicle.

As a result of the determination processing in step S1404, if it is determined that the first image and the second image are images captured by the same vehicle (step S1404: Yes), the third device 1301 The device 302 is determined to be the target device of cooperative control, and the determination result is notified to the first device 300 and the second device 302 (step S1405). After that, the third device 1301 cooperatively controls the first device 300 and the second device 302 . As a result of the determination processing in step S1404, if the first image and the second image are not determined to be images shot by the same vehicle (step S1404: No), the process returns to step S1403.

The first device 300 and the second device (information device 2) 302 wait for the notification of the determination result of the identity of the photographing positions from the third device 1301 (step S1413). When the third device 1301 determines that the shooting positions of the first image and the second image are the same, the first device 300 and the second device 302 are switched from the third device 1301 to the first device 300 and the second device. 302 are images taken by the same vehicle (step S1414) and registered (step S1415). As this notification, it may be notified that the first device 300 is mounted in the same vehicle as the second device 302 .

Also, the third device 1301 transmits a synchronization (linkage) confirmation screen to the user as shown in FIG. 300 and the second device 302 can also be synchronized (coordinated).

After that, the first device 300 and the second device 302 are under cooperative control by the third device 1301 . Linkage control can link the displays of the first device 300 and the second device 302 in the same manner as in the first embodiment (eg, FIGS. 9 to 12).

According to the second embodiment described above, by providing the function of the determination device 100 in the third device (server) 1301 external to the first device 300 and the second device 302, the third device 1301 can When the second device 302 is detected, it is determined whether the first device 300 and the second device 302 are devices provided in the same vehicle. At this time, the third device 1301 determines by image processing whether the first image and the second image are images shot from the same vehicle. The third device 1301 acquires the first image of the first device 300 and the second image of the second device 302, compares the first image and the second image, and obtains the first image and the second image. It can be determined whether the equipment 302 is equipment provided in the same vehicle.

As a result, for example, when the second device 302 is newly installed after the first device 300 has already been installed, there is no need to perform complicated user settings indicating that the second device 302 has been installed. The device 302 can now be controlled in cooperation. Note that the first device 300 and the second device 302 are not limited to two devices, and may be a plurality of devices. can.

In each of the above embodiments, a navigation device is used as the first device and a mirror-type terminal is used as the second device. For example, a smart phone, a tablet, a PC, or the like may be used.

In addition, in each embodiment, the first device and the second device hold the activation time (for example, the activation time by ACC ON of the vehicle), and the device having the function of the determination device (the first device in the first embodiment or the implementation The activation time may be transmitted to the third device in Example 2). A device having the function of a determination device may determine the identity of the photographing positions of the first image and the second image, including this activation time. As a result, it is possible to prevent erroneous identity determination for devices in different vehicles.

For example, it is conceivable that a device having the function of a determination device acquires an image captured by another vehicle traveling in the next lane. In this case, when the image captured by the other vehicle is compared with the image captured by the vehicle, the determination device captures the same subject at the same time in the image captured by the other vehicle and the image captured by the vehicle. There is a risk of erroneous determination that there is location identity (photographed by the same vehicle).

For this reason, when judging the identity of shooting positions using the first image and the second image, by including the startup time in the judgment, the startup of the first device (own vehicle) and the second device (another vehicle) can be minimized. If the times do not match (for example, they differ by 10 minutes or more), it can be determined that the first image and the second image are not images captured by the same vehicle equipment. As a result, even if it is determined that the photographing positions of the first image and the second image are the same, it can be determined that they are not actually the same vehicle by image processing alone.

The determination method described in this embodiment can be realized by executing a prepared program on a computer such as a personal computer or a workstation. This program is recorded on a computer-readable recording medium such as a hard disk, flexible disk, CD-ROM, MO, DVD, etc., and is executed by being read from the recording medium by a computer. Also, this program may be a transmission medium that can be distributed via a network such as the Internet.

REFERENCE SIGNS LIST 100 determination device 101 first acquisition unit 102 second acquisition unit 103 determination unit 104 display control unit 105 output unit 300 navigation device (first device)
302 information equipment (second equipment)
311, 312 Camera 1301 Server (third device)

Claims (1)

a first acquisition means for acquiring a first image captured by the first device;
a second acquisition means for acquiring a second image captured by the second device;
determination means for determining identity between the first image and the second image based on the captured images of the first image and the second image;
output means for outputting to the first device or the second device that the first device and the second device can cooperate with each other according to the determination result by the determination means;
A determination device characterized by comprising:

Images