JP5566443B2 - Network camera system - Google Patents

Description

The present invention relates to a network camera system, and more particularly to a network camera system suitable for an imaging apparatus arranged in a local area network.

In recent years, terminal devices such as video cameras having a network connection function and personal computers having an image capturing function have been provided as image capturing apparatuses. An image captured by such an imaging apparatus can be viewed from a personal computer or a mobile phone located at a remote location using the Internet or the like. An example of technology related to such network cameras is
It is described in Patent Document 1.

Some imaging devices and the like have a wired or wireless local area network (LAN) connection function.

Japanese Patent No. 4095395

By the way, when an imaging device or the like as described above is installed in a house or the like and constitutes a LAN together with other devices such as a personal computer, a private IP address (local IP address) is assigned to the imaging device or the like instead of a global IP address. Because there are many cases, so-called NAT
(Network Address Translation) Crossing problem occurs. That is, there is a problem that the user cannot access an imaging device or the like in the local network from a mobile terminal or the like belonging to the external network, and it is difficult to view and acquire the captured image from the outside. is there. In addition, there is a need among users to browse images captured by an imaging device or the like using a device or communication mode desired by the user.

Accordingly, an object of the present invention is to provide communication between an imaging apparatus and the like that belong to a local network and are not assigned a fixed global IP address and an external network.

It is another object of the present invention to provide a network camera system that can arbitrarily browse an image captured by an imaging device that belongs to a local network and is not assigned a fixed global IP address from an external network. .

The present invention also provides a network camera system capable of browsing an image captured by an imaging device or the like that belongs to a local network and is not assigned a fixed global IP address, using a device or communication mode desired by a user. The purpose is to provide.

The server of the present invention is a server configured to be able to communicate with an imaging device belonging to a local area network and a terminal device belonging to a network other than the local area network, and is periodically transmitted from the imaging device. Means for receiving an inquiry about the presence or absence of a connection request for the imaging device, and when receiving an inquiry from the imaging device, determines whether a connection request for the imaging device is received from the terminal device; Means for maintaining a connection with the imaging apparatus established based on the inquiry on the condition that the determination result is right, and the imaging transmitted from the imaging apparatus using the maintained connection Means for receiving imaging data captured by the apparatus; means for transmitting the received imaging data to the terminal apparatus; Characterized in that it comprises.

The server of the present invention is a server configured to be able to communicate with a first terminal device belonging to a local area network and a second terminal device belonging to a network other than the local area network, Means for receiving an inquiry about the presence or absence of a connection request for the first terminal device periodically transmitted from the terminal device; and when receiving an inquiry from the first terminal device, from the second terminal device It is determined whether or not a connection request for the first terminal device has been received, and the connection with the first terminal device established based on the inquiry on condition that the determination result is correct , Means for receiving data transmitted from the first terminal device using the maintained connection, and the received first terminal device Characterized in that it comprises means for transmitting data to be al transmitted to the second terminal apparatus.

The server of the present invention is a server configured to be able to communicate with an imaging device belonging to a local area network and a terminal device belonging to a network other than the local area network, and the terminal device transmitted from the terminal device Means for receiving the identification information of the imaging device, and the identification information of the imaging device and the I of the imaging device that are periodically transmitted from the imaging device.
Established on the basis of the received IP address of the imaging device, when the means for receiving the P address and the received identification information of the terminal device are associated in advance with the received identification information of the imaging device Connection maintaining means for maintaining a connection with the imaging apparatus, means for receiving data taken by the imaging apparatus transmitted from the imaging apparatus using a connection maintained by the connection maintaining means, Means for transmitting received data taken by the imaging device to the terminal device.

The server of the present invention is a server configured to be able to communicate with a first terminal device belonging to a local area network and a second terminal device belonging to a network other than the local area network, and the second terminal device Means for receiving identification information of the second terminal device transmitted from the terminal device, and the first terminal device transmitted at regular intervals from the first terminal device.
Means for receiving the identification information of the first terminal device and the IP address of the first terminal device, and the received identification information of the second terminal device are associated in advance with the received identification information of the first terminal device. Connection maintaining means for maintaining a connection with the first terminal apparatus established based on the received IP address of the first terminal apparatus, and a connection maintained by the connection maintaining means Means for receiving the data transmitted from the first terminal device using the method, and means for transmitting the received data transmitted from the first terminal device to the second terminal device. It is characterized by providing.

The server of the present invention is a server configured to be able to communicate with an imaging device belonging to a local area network and a mobile phone terminal belonging to a network other than the local area network and having a videophone function, and the mobile phone terminal First storage means for storing the identification information of the imaging device and the identification information of the imaging device in association with each other, and the identification information of the imaging device and the IP
A second storage means for storing the address and the port number in association with each other; and the identification information, the IP address, and the port number of the imaging apparatus that are periodically transmitted from the imaging apparatus; When receiving the videophone call from the mobile phone terminal and the update means for registering and updating the corresponding information in the storage means, the identification information of the imaging device corresponding to the identification information of the mobile phone terminal is stored in the first storage. First extraction means for extracting from the means; second extraction means for extracting from the second storage means an IP address and port number corresponding to the identification information of the imaging device extracted by the first extraction means; Based on the IP address and port number extracted by the second extraction means, transfer means for transferring the outgoing videophone call to the imaging device, and the videophone call transferred by the transfer means Corresponding to the communication, comprising: means for receiving the moving image / sound data captured by the imaging device transmitted from the imaging device; and means for transmitting the received moving image / sound data to the mobile phone terminal. Features.

The server of the present invention is a server configured to be able to communicate with an imaging device belonging to a local area network and an information processing terminal belonging to a network other than the local area network, and the domain name and IP address of the imaging device First storage means for storing the information in association with each other, second storage means for storing the identification information, IP address and port number of the image pickup apparatus in association with each other, and periodically transmitted from the image pickup apparatus, First update means for receiving the domain name and IP address of the imaging device and registering and updating the corresponding information in the first storage means, and identification information of the imaging device periodically transmitted from the imaging device A second updating means for receiving the IP address and port number and registering and updating the corresponding information in the second storage means; and the information processing When a browsing request including a domain name is received from the end, a first extraction unit that extracts an IP address corresponding to the domain name from the first storage unit, and an IP address extracted by the first extraction unit Second extraction means for extracting the port number to be extracted from the second storage means, the IP address extracted by the first extraction means, and the port number extracted by the second extraction means are transmitted to the information processing terminal And means for performing.

In this specification and the like, the means does not simply mean a physical means, but includes a case where the functions of the means are realized by software. Further, the function of one means may be realized by two or more physical means, or the functions of two or more means may be realized by one physical means.

The present invention is not limited to the above-described embodiment, and can be implemented in various other forms without departing from the gist of the present invention. For this reason, the said embodiment is only a mere illustration in all points, and is not interpreted limitedly. For example, the above-described processing steps can be executed in any order or in parallel as long as there is no contradiction in the processing contents.

According to the present invention, communication between an imaging device or the like that belongs to a local network and is not assigned a fixed global IP address can be performed. Further, according to the present invention, an image captured by such an imaging apparatus or the like can be arbitrarily browsed from an external network. Further, according to the present invention, the user can view the captured image using the device or communication mode desired by the user.

It is a figure explaining the whole structure of the image delivery system which concerns on 1st Embodiment. It is a flowchart which shows the flow of the image delivery process which concerns on 1st Embodiment. It is a block diagram showing the structure of the network camera system which concerns on 2nd Embodiment. It is a block diagram showing the structure of the imaging device which concerns on 2nd Embodiment. It is a block diagram showing the structure of the relay server which concerns on 2nd Embodiment. It is a block diagram showing the structure of the gateway which concerns on 2nd Embodiment. It is a figure explaining the structure of the portable terminal which concerns on 2nd Embodiment. It is a figure explaining the flow of the NAT traversal process which concerns on 2nd Embodiment. It is a figure explaining the whole network camera system composition concerning a 3rd embodiment. It is a block diagram showing the structure of the imaging device which concerns on 3rd Embodiment. It is a figure explaining the flow of the NAT traversal process which concerns on 3rd Embodiment.

(1) First Embodiment Hereinafter, a network camera system according to a first embodiment will be described in detail. FIG. 1 is a configuration diagram showing the overall configuration of an image distribution system 1 as a network camera system. The imaging device (terminal device) 5 is, for example, a network camera, installed in a facility such as a home or a company, and is connected to the information processing terminal 6, the information processing terminal 7, the hub 8, and the router 9 together with the L.
An AN4 (Local Area Network) is configured. The terminal device 3, the relay server 2, and the recording / billing server 10 are configured to be able to communicate with each other via a communication network N (including wired and wireless), and are also configured to be able to communicate with each device in the LAN 4. .

A private IP address (local IP address) is assigned to the imaging device 5, the information processing terminal 6, and the information processing terminal 7 that constitute the LAN 4.
Connection with the external network of the AN 4 is performed via the broadband router 9 to which a global IP address is assigned. The broadband router 9 is a private I
NAT (Network Address) that transparently translates P address and global IP address
Translation) function, or NAPT that translates port numbers in addition to IP addresses
(Network Address Port Translation) function is provided.

In the present embodiment, the mobile terminal 3 having a camera function is used as the terminal device 3 operated by the user.
1, a digital video recorder 32 and a personal computer 33 are illustrated in FIG. However, the terminal device 3 should just be an apparatus which can transmit a connection request to the relay server 2, and is not restricted to these.

In such a network environment, when the user wants to browse a video of a subject (for example, a pet at home) photographed by the imaging device 5, the user operates the terminal device 3 to request browsing of the subject video. Transmit to the relay server 2. However, the relay server 2 cannot access the imaging device 5 belonging to the LAN 4 and assigned with a private IP address from the outside.

Therefore, in the present embodiment, the imaging device 5 periodically inquires of the relay server 2 belonging to the external network whether there is a browsing request for the terminal device 3, and the relay server 2 does not receive a browsing request from the terminal device 3. Releases the connection (session) with the imaging device 5 each time, and maintains the connection when there is a browsing request from the terminal device 3. The imaging device 5 includes the relay server 2
The subject is photographed in a state where the connection is maintained, and the photographed data is transmitted to the relay server 2. As described above, the relay server 2 establishes and maintains a connection with the imaging device 5 in response to a user's browsing request, and uses the maintained connection to obtain shooting data from the imaging device 5 in real time. This is received and transmitted to the terminal device 3.

According to such an image distribution system 1, the user of the imaging device 5 accesses the relay server 2 using the terminal device 3, so that the subject image captured by the imaging device 5 can be viewed in real time from a remote location. You can browse at.

FIG. 2 is a flowchart showing a flow of image distribution processing by the image distribution system 1 according to the present embodiment. Each step (including a partial step to which no reference numeral is assigned) in the flowchart described below can be executed in any order or in parallel within a range that does not contradict the processing contents.

Prior to the use of the image distribution service by the image distribution system 1, the user accesses the relay server 2 from the terminal device 3 and transmits user registration information (S100). The user registration information includes identification information for uniquely identifying the user's terminal device 3 (for example, a terminal device ID and a mobile phone number) and identification information for uniquely identifying the user's imaging device 5 ( For example, imaging device ID) is included.

For example, the user captures a barcode (not shown) attached to the imaging device 5 with the camera of the terminal device 3, and transmits the captured barcode image together with the terminal device ID to the relay server 2. When the relay server 2 receives the user registration information, the relay server 2 associates the received terminal device ID with the imaging device ID (obtained by converting the barcode image) and stores it in a predetermined table (S102). Thereby, in the relay server 2, the user's imaging device 5 and the terminal device 3 are managed in association with each other.

When the imaging apparatus 5 is powered on (S104), a predetermined program (NAT traversing module) stored in the memory is executed by the CPU, and the relay server 2 is connected via the router 9.
An inquiry about the presence / absence of a connection request is sent to (S106). The inquiry about whether or not there is a connection request is for confirming whether or not there is a connection request inquiry (request for acquiring / browsing photographing data of the image pickup device 5) from the terminal device 3. This inquiry includes an imaging device ID, a private IP address assigned to the imaging device, and a port number.

When the router 9 accepts the inquiry, the router 9 refers to the mapping table that defines the correspondence between the IP address and the port number, and assigns the private IP address and the port number of the imaging device 5 included in the inquiry to the corresponding global IP address The address and other port number are converted and sent to the communication network N. Similarly, when data destined for the imaging device 5 is received from the communication network N, refer to the mapping table,
Private IP corresponding to global IP address and port number in the data
The address is converted into another port number and sent to the LAN 4.

When the relay server 2 receives an inquiry about whether or not there is a connection request from the imaging device 5, the private IP address and port number of the imaging device 5 included in the inquiry (the global IP address and port number after conversion when viewed from the relay server) ) And T between the imaging device 5
Establish a connection based on CP / IP.

Then, the connection request information of the terminal device 3 having the terminal device ID associated with the imaging device ID is referred from a predetermined table (connection request management table). Since OFF is set as a default in the connection request information, the relay server 2 returns connection request information (OFF) (S110). Then, since (OFF) is set in the connection request information, the relay server 2 releases the connection with the imaging device 5 (S112).

When receiving the connection request information (OFF) from the relay server 2, the imaging device 5 continuously transmits an inquiry as to whether or not there is a connection request according to a predetermined interval (S114). The fixed interval can be set arbitrarily according to the specification or by the user. In this embodiment, 5 seconds is set as an example.

On the other hand, when the user wants to browse the video of the subject captured by the imaging device 5, the user operates the terminal device 3 to transmit a connection request (acquisition data acquisition request of the imaging device 5) to the relay server 2. (S116). The connection request includes the terminal device ID.

When the relay server 2 receives the connection request from the terminal device 3 (S118), the relay server 2 refers to a predetermined table (connection request management table) and sets “ON” in the connection request information corresponding to the terminal device ID included in the connection request. Set (S120).

Note that after the transmission in S106, the imaging device 5 transmits an inquiry as to whether or not there is a connection request to the relay server 2 via the router 9 at regular intervals (5 seconds) (S122). The inquiry includes an imaging device ID, a private IP address assigned to the imaging device, and a port number. Note that after the conversion by the router 9, the imaging device ID, the global IP address, and the port number are included.

When the relay server 2 receives an inquiry about the presence or absence of a connection request from the imaging device 5, the private IP address and port number of the imaging device included in the inquiry (the global IP address and port number after conversion when viewed from the relay server) TC with the imaging device 5 using
Establish a connection based on P / IP.

Then, the connection request information of the terminal device 3 corresponding to the imaging device ID included in the inquiry is referred from a predetermined table (connection request management table). Here, since ON is set in the connection request information, the relay server 2 returns the connection request information (ON) (S126).
Then, since (ON) is set in the connection request information, the relay server 2 maintains the connection established with the imaging device 5 based on TCP / IP (S128).

When receiving the connection request information (ON) from the relay server 2, the imaging device 5 starts a data transmission process (S130). The imaging device 5 captures the subject with the camera mechanism (S13).
2) The photographed image data (still image, moving image, and / or audio) is transmitted to the relay server 2 using the maintained connection (session) (S134). UDP / IP can be used for transmission of imaging data. Further, shooting of the subject and transmission of the shooting data can be performed according to specifications and settings. For example, a still image may be transmitted at a predetermined interval or a moving image may be transmitted.

When the relay server 2 receives the image data (S136), the relay server 2 performs predetermined image processing on the received image data and transmits the image data to the terminal device 3 (S138). When the terminal device 3 receives the shooting data, it outputs it to the display (S140).

Further, the relay server 2 transmits the shooting data in association with the terminal device ID or the imaging device ID to the recording / billing server 10 (S142), and the recording / billing server 10 transmits the received shooting data to the terminal device ID or The information is stored in a predetermined storage area in association with the imaging device ID (S144). The recording / billing server 10 executes a billing process for the use of the shooting data in accordance with predetermined setting conditions.

According to the first embodiment, communication between the imaging device 5 belonging to the local area network and not assigned with the global IP address and the relay server 2 belonging to the external network becomes possible.

Also, only when there is a connection request from the terminal device 3, the connection between the imaging device 5 and the relay server 2 is maintained, and shooting data is transmitted from the imaging device 5 to the relay server 2. Therefore, the connection between the imaging device 5 and the relay server 2 is efficiently maintained, and the load on the relay server 2 can be reduced.

In the first embodiment, the case where there is an acquisition request for the imaging data from the terminal device 3 to the imaging device (terminal device) 5 has been described as an example. However, the configuration of the first embodiment is as follows. However, the present invention is not limited to this. For example, the same applies to a case where the terminal device 3 makes an acquisition request for data held by the information processing terminal 6 or the information processing terminal 7.

In the first embodiment, the server is a server configured to be able to communicate with a terminal device, an imaging device belonging to a local area network, and the first connection request transmitted from the imaging device at regular intervals. , Receiving means for receiving a second connection request transmitted from the terminal device, and determining whether or not the second connection request has been received when the first connection request is received If it is determined that the second connection request has been received, a connection maintaining unit that maintains a connection with the imaging apparatus established based on the first connection request, and a connection maintaining unit that maintains the connection. Means for receiving data photographed by the imaging device transmitted from the imaging device using a connected connection, and means for transmitting data received by the imaging device to the terminal device.

The server is a server configured to be able to communicate with the first terminal device, the second terminal device belonging to the local area network, and transmitted from the second terminal device at regular intervals. Means for receiving the second connection request, means for receiving the second connection request transmitted from the first terminal device, and receiving the second connection request when the first connection request is received. If it is determined whether or not the second connection request has been received, the connection maintenance for maintaining the connection with the second terminal device established based on the first connection request Means,
Means for receiving data transmitted from the second terminal apparatus using the connection maintained by the connection maintaining means, and data received by the second terminal apparatus received by the first terminal apparatus.
Means for transmitting to the terminal device.

Further, the server is a server configured to be able to communicate with a terminal device, an imaging device belonging to a local area network, means for receiving identification information of the terminal device transmitted from the terminal device, and the imaging Means for receiving the identification information of the imaging device and the private IP address of the imaging device, which are transmitted from the device at regular intervals, and the received identification information of the terminal device is the same as the identification information of the received imaging device in advance. A connection maintaining unit that maintains a connection with the imaging device established based on the private IP address of the received imaging device, and a connection maintained by the connection maintaining unit Means for receiving data taken by the imaging device transmitted from the imaging device, and receiving the data taken by the imaging device. And means for transmitting to the terminal device.

The server is a server configured to be able to communicate with the first terminal device, the second terminal device belonging to the local area network, and transmitted from the first terminal device. Means for receiving identification information and transmitted from the second terminal device at regular intervals;
Means for receiving the identification information of the second terminal device and the private IP address of the second terminal device; and the received identification information of the first terminal device is the identification of the received second terminal device. A connection maintaining means for maintaining a connection with the second terminal device established based on the received private IP address of the second terminal device when the information is associated with the information in advance, and the connection maintaining Means for receiving data transmitted from the second terminal device using a connection maintained by the means, and the received data transmitted from the second terminal device to the first terminal device Means for transmitting.

(2) Second Embodiment FIG. 3 is a block diagram showing a schematic configuration of a network camera system according to a second embodiment. In the second embodiment, a case where a captured image of a network camera is viewed using a mobile TV phone will be described. In FIG. 3, the same components as those in FIG. 1 are denoted by the same reference numerals, and description thereof is omitted here.

The network camera system according to the second embodiment is mainly characterized in that the terminal device is a mobile phone having a TV phone function, and that a gateway server 11 is provided for receiving a TV phone call from the mobile phone. . The network camera 51 as an imaging device includes a LAN 4 (Local Area) together with the information processing terminal 6, the information processing terminal 7, the hub 8 and the router 9.
Network). The network camera 51 and the mobile phone 34 correspond to the communication network N2 (for example, but not limited to the ISDN line), the gateway server 11 and the communication network N (for example, the Internet, but are not limited thereto). Are not configured to communicate with each other. In FIG. 3, one mobile phone 34
However, one or more mobile phones 34 can be connected to the network camera system. In FIG. 3, the relay server 21 and the gateway server 11 are described separately, but these functions may be configured by a single computer.

FIG. 4 is a block diagram illustrating a functional configuration of the network camera according to the second embodiment. The network camera 51 includes a communication unit (communication unit) 511, a control unit (control unit) 512,
A storage unit (storage unit) 513, a voice input / output unit (voice input / output unit) 514, an imaging unit (imaging unit) 515, a display unit (display unit) 516, and an operation unit (operation unit) 517 are provided. Although not shown, the network camera 51 includes hardware including a CPU, a memory, an external storage device, an input / output interface, a communication interface, a camera mechanism, and a bus connecting these, for example, the CPU stores in the memory or the storage device. The above-described functional units are realized by executing the various programs and controlling the hardware.

The communication unit 511 controls communication so as to input / output information to / from the network camera 51 via the LAN 4, and includes a module such as TCP / IP. The control unit 512 controls the overall operation of the network camera 51, and includes a NAT traversal processing unit that performs NAT traversal processing, a videophone processing unit that performs videophone processing, and the like. The storage unit 513 stores various programs and data necessary for NAT traversal processing as appropriate. Here, a camera ID that uniquely identifies a network camera is stored as user registration information. The voice input / output unit 514 controls voice input / output with respect to the network camera 51, and corresponds to, for example, a speaker, a microphone, an earphone, and the like. The imaging unit 515 captures a subject by, for example, a CCD (Charge Coupled Devices) image sensor, and generates a captured image by appropriately performing A / D conversion processing, image processing, or the like on the imaging result (image signal). The display unit 516 is for displaying a captured image or the like, and corresponds to, for example, a liquid crystal display. The operation unit 517 receives user operations, and corresponds to, for example, a switch, a button, an operation key, a keyboard, and the like. Note that the configuration and functions of the network camera 51 are not limited to these, and functions can be added or changed according to settings.

FIG. 5 is a block diagram illustrating the configuration of the relay server according to the second embodiment. The relay server 21 includes a communication unit 211, a control unit 212, a storage unit 213, an input unit 214, and an output unit 215. The relay server 21 may be composed of a single computer or may be composed of a plurality of computers distributed on a network. Although not shown, the relay server 21 includes hardware including a CPU, a memory, an external storage device, an input / output interface, a communication interface, and a bus connecting these, for example, CP
Various functions are realized by U executing various programs stored in a memory or a storage device.

The communication unit 211 controls communication so as to input / output information to / from the relay server 21 via the communication network N, and includes a module such as TCP / IP, for example. The control unit 212 controls the operation of the entire relay server 21 and includes a camera information update unit that appropriately updates camera information, a determination unit that determines an access destination (network camera) for NAT traversal, and the like. The storage unit 213 appropriately stores various programs and data necessary for NAT traversal processing and the like. Here, a camera ID that uniquely identifies the network camera 51 and an IP address assigned to the network camera 51. And the port number are stored in association with each other as user registration information. The input unit 214 and the output unit 215 are for inputting / outputting various data, and correspond to, for example, a keyboard and a display.
Note that the configuration and functions of the relay server 21 are not limited to these, and functions can be added or changed according to settings.

FIG. 6 is a block diagram showing the configuration of the gateway server according to the second embodiment. The gateway server 11 includes a communication unit 111, a control unit 112, a storage unit 113, an input unit 114,
An output unit 115 is provided. The gateway server 11 may be composed of a single computer or may be composed of a plurality of computers distributed on a network. Although not shown, the gateway server 11 includes a CPU, a memory,
It includes hardware including an external storage device, an input / output interface, a communication interface, and a bus connecting them, and for example, the CPU implements various functions by executing various programs stored in the memory or the storage device.

The communication unit 111 controls communication so as to input / output information to / from the gateway server 11 via the communication network N and the communication network N2, and includes a module such as TCP / IP or SIP, for example. The control unit 212 controls the overall operation of the gateway server 11, and includes a user information registration unit that registers user information, an NA
An inquiry unit for inquiring about an access destination (network camera) for exceeding T, a TV phone relay unit for relaying a video phone, and the like are provided. The storage unit 113 appropriately stores various programs and data necessary for NAT traversal processing and the like.
Are stored in association with the camera ID uniquely identifying the network camera 51. The input unit 114 and the output unit 115 are for inputting / outputting various data, and correspond to, for example, a keyboard and a display. Note that the configuration and functions of the gateway server 11 are not limited to these, and functions can be added or changed according to settings.

FIG. 7 is a diagram illustrating the configuration of the mobile phone according to the second embodiment. Mobile phone 34
Are a communication unit 341, a control unit 342, a storage unit 343, a voice input / output unit 344, an imaging unit 345,
A display unit 346 and an operation unit 347 are provided. Although not shown, the mobile phone 34 is a CPU.
, A memory, an external storage device, an input / output interface, a communication interface, and hardware including a bus connecting them. For example, the CPU implements various functions by executing various programs stored in the memory or the storage device.

The communication unit 341 controls communication so as to input / output information to / from the mobile phone 34 via the communication network N2, and includes a module such as SIP. The control unit 342 controls the operation of the mobile phone 34 as a whole, and includes a TV phone processing unit that executes processing related to the video phone. The storage unit 343 stores various programs and data as appropriate. Here, the mobile phone number of the mobile phone 34 is stored. The voice input / output unit 344 controls voice input / output with respect to the mobile phone 34, for example, a speaker,
This applies to microphones and earphones. The imaging unit 345 is, for example, a CCD (Charge Coupled Dev).
ices) A subject is imaged by an image sensor, and A / D is performed on the imaging result (image signal).
A captured image is generated by appropriately performing conversion processing, image processing, or the like. The display unit 346 is for displaying a captured image or the like, and corresponds to, for example, a liquid crystal display. Operation unit 347
Is for accepting a user operation, and corresponds to, for example, a button or an operation key. Note that the configuration and functions of the mobile phone 34 are not limited to these, and functions can be added or changed according to settings.

Next, the process flow of the network camera system according to the second embodiment configured as described above will be described in detail. FIG. 8 is a flowchart showing a flow of NAT traversal processing according to the second embodiment. Each step (including a partial step to which no reference numeral is assigned) in the flowchart described below can be executed in any order or in parallel within a range that does not contradict the processing contents.

Prior to using the service by the network system, the user accesses the gateway server 11 and the relay server 21 from the mobile phone 34, and transmits user registration information (S200). The user registration information addressed to the gateway server 11 includes identification information for uniquely identifying the user's mobile phone 34 (for example, a terminal ID or a telephone number registered in the SIM card of the mobile phone 34), Identification information (for example, camera ID) for uniquely identifying the network camera 51 is included. The user registration information addressed to the relay server 21 includes at least a camera ID.

Upon receiving the user registration information, the gateway server 11 and the relay server 21 store the received user registration information in predetermined storage areas (S202 and S204).

On the other hand, when the power is turned on, the control unit 512 of the network camera 51 executes the NAT traversal module and periodically transmits information (camera information) about the network camera 51 to the relay server 2 via the router 9 (S206). . The camera information includes a camera ID, a private IP address assigned to the network camera 51, and a port number (port number).
including. The timing for transmitting the camera information can be appropriately set according to the design. Note that the dotted line surrounding the processing steps related to S206, S208, and S210 in FIG.
6, S208 and S210 are repeatedly executed until a predetermined condition (for example, until the power is turned off) is satisfied.

The camera information sent from the network camera 51 is sent to the relay server 21 after the IP address and port number are converted by the router 9 (S208).

When the relay server 21 acquires the camera information, the relay server 21 associates the camera ID, the global IP address, and the port number included in the camera information and stores them in a predetermined storage area (S210).
. If the camera information has already been stored, the relay server 21 identifies the corresponding data based on the camera ID, and registers and updates the identified data. As a result, the relay server 21
Holds the latest IP address and port No. of the network camera 51.

On the other hand, when the user wants to view the captured image of the network camera 51, the mobile phone 3
4 is operated to make a videophone call, thereby making a videophone call to the gateway server 11 (S212). The gateway server 11 receives a videophone call from the mobile phone 34 (S214).

When receiving the videophone call, the gateway server 11 specifies the mobile phone number of the mobile phone 34 that is the originator of the videophone based on, for example, a caller ID notification function (S21).
6) The camera ID corresponding to the mobile phone number is specified by referring to the user registration information using the specified mobile phone number as a key (S216).

Based on the identified camera ID, the gateway server 11 inquires the relay server 21 about the destination of the videophone call, that is, the access destination (network camera) (S218).
.

The relay server 21 specifies access destination information (global IP address and port No.) corresponding to the camera ID by referring to the camera information using the camera ID included in the received inquiry as a key (S220).

The relay server 21 specifies the specified access destination information (global IP address and port number).
Is sent back to the gateway server 11 (S222).

The gateway server 11 transmits the transmission information of the mobile phone 34 and the received access destination information (global IP address and port number) to the network camera 51 via the router 9 (S226).

The router 9 converts the destination address included in the videophone request according to a predetermined mapping table (global IP address → private IP address, port No → other port No) and sends it to the LAN 4 (S228).

When the network camera 51 receives a videophone request (S230), the network camera 51 starts a videophone process and starts transmitting moving image audio data (captured image) (S232).

The router 9 converts the source address included in the video / audio data according to a predetermined mapping table (private IP address → global IP address, other port No → port No), and sends it to the communication network N (S234). .

The video / audio data is transmitted via the gateway server 11 (S236) to the mobile phone 34.
The mobile phone 34 receives and reproduces the moving image audio data (S238). FIG.
The processing steps according to S232, S234, and S236 are repeatedly executed until a predetermined condition (for example, until a videophone end instruction is received) is satisfied.

When the user performs an operation for terminating the videophone on the cellular phone 34, the cellular phone 34 transmits an instruction to terminate the videophone to the network camera 51 (S24).
0). The videophone end instruction is transmitted to the network camera 51 via the gateway server 11 (S242).

The router 9 converts the destination address included in the videophone end instruction according to a predetermined mapping table (global IP address → private IP address, port No → other port No) and sends it to the LAN 4 (S244).

When the network camera 51 receives the videophone end instruction, the network camera 51 stops the transmission of the moving image audio data (S246).

In the above flow, the case where the mobile phone 34 receives and reproduces the moving image audio data captured by the network camera 51 has been described. At the same time, the moving image audio data received the moving image audio data captured by the mobile phone 34. You may make it reproduce | regenerate.

As described above, according to the second embodiment, in addition to the same effects as those of the first embodiment, the user browses the captured image of the network camera using the communication mode desired by the user, such as the videophone function. Will be able to. In addition, by using the videophone function, it is possible to view the captured image of the network camera in real time.

(3) Third Embodiment FIG. 9 is a block diagram illustrating a schematic configuration of a network system according to a third embodiment. In the third embodiment, a case where a captured image of a network camera is browsed using a DDNS server will be described. In FIG. 9, the same components as those in FIG. 1 are denoted by the same reference numerals, and description thereof is omitted here.

As shown in FIG. 9, the network camera system according to the third embodiment has a DDNS (
It is characterized in that a captured image is viewed from a terminal device such as a personal computer using a Dynamic Domain Name System) server. The network camera 52 as an imaging device includes an information processing terminal 6, an information processing terminal 7, a hub 8 and a router 9 together with a LAN 4 (Local Area
work). The network camera 52, the relay server 22, and the terminal device 3 are connected via a communication network N (for example, the Internet is applicable, but is not limited thereto).
It is configured to be able to communicate with each other. The DDNS server 12 has a function of dynamically registering and managing the correspondence between the dynamically assigned IP address and its host name. In FIG. 9, the relay server 21 and the DDNS server 12 are described separately, but these functions may be configured by a single computer.

FIG. 10 is a block diagram illustrating a functional configuration of a network camera according to the third embodiment. The network camera 52 includes a communication unit 521, a control unit 522, a storage unit 523, a voice input / output unit 524, an imaging unit 525, a display unit 526, and an operation unit 527. The network camera shown in FIG. 10 is different from the control unit of the network camera shown in FIG. 4 in that the control unit includes a DDNS processing unit instead of the videophone processing unit, but other configurations are common.
As will be described later, the DDNS processing unit periodically transmits the domain name and the IP address to the DDNS server 12.

Next, the process flow of the network camera system according to the third embodiment configured as described above will be described in detail. FIG. 11 is a flowchart showing a flow of NAT traversal processing according to the third embodiment. Each step (including a partial step to which no reference numeral is assigned) in the flowchart described below can be executed in any order or in parallel within a range that does not contradict the processing contents.

When the power is turned on, the control unit 522 of the network camera 52 executes the NAT traversal module and periodically transmits information (camera information) about the network camera 52 to the relay server 22 via the router 9 (S300). The camera information includes a camera ID, a private IP address assigned to the network camera 51, and a port number. The timing for transmitting the camera information can be appropriately set according to the design. In addition, the dotted line surrounding the process steps according to S300 to S310 in FIG. 8 indicates that the process steps according to S300 to S310 are repeatedly executed until a predetermined condition (for example, until the power is turned off) is satisfied. Yes.

The camera information sent from the network camera 52 is sent to the relay server 22 after the IP address and port number are converted by the router 9 (S302).

When the relay server 22 acquires the camera information, the relay server 22 associates the camera ID, the global IP address, and the port number included in the camera information and stores them in a predetermined storage area (S304).
. If the camera information has already been stored, the relay server 22 identifies the corresponding data based on the camera ID, and registers and updates the identified data. As a result, the relay server 22
Holds the latest IP address and port number of the network camera 52.

Similarly, the control unit 522 of the network camera 52 executes the DDNS module, and periodically transmits information (domain information) about the domain of the network camera 52 to the relay server 22 via the router 9 (S306). The domain information includes a domain name and a private IP address assigned to the network camera 51. The timing for transmitting the domain information can be appropriately set according to the design.

The domain information sent from the network camera 52 is converted into an IP address and a port number by the router 9 and then sent to the DDNS server 12 (S308).

When acquiring the domain information, the DDNS server 12 associates the domain name with the global IP address and stores them in a predetermined storage area (S310). If the domain name is already stored, the relevant data is identified and updated based on the domain name. As a result, the DDNS server 12 holds the latest IP address and domain name association information of the network camera 52.

On the other hand, the user inputs a domain name from the terminal device 3 and transmits a browsing request (S311).
The DDNS server 12 receives the browsing request (S312).

The DDNS server 12 specifies a global IP address corresponding to the domain name included in the browsing request from a predetermined storage area (S314), and transfers the browsing request based on the specified global IP address (S316).

The relay server 22 identifies the camera ID and the port number corresponding to the global IP address by referring to the camera information using the global IP address included in the browsing request as a key (S318).

The relay server 22 identifies the specified access destination information (global IP address and port number).
Is sent back to the terminal device 3 (S320).

The terminal device 3 obtains the acquired access destination information (global IP address and port number) (S
322), an imaging data acquisition request is transmitted (S324).

The imaging data acquisition request is made after address conversion is performed via the router 9 (S326).
Is transmitted to the network camera 52 (S328).

Upon receiving the imaging data acquisition request (S328), the network camera 52 starts imaging processing and starts transmitting captured moving image audio data (captured image) (S330).

The transmitted captured moving image audio data (captured image) is address-converted via the router 9 (S332), and then transmitted to the terminal device 3.

The terminal device 3 receives the moving image audio data (captured image) and reproduces it (S334).
. The processing steps according to S330, S332, and S334 in FIG. 11 are repeatedly executed until a predetermined condition (for example, until an acquisition end instruction is received) is satisfied.

When the user performs an operation to give an acquisition end instruction to the terminal device 3, the terminal device 3 transmits an imaging data acquisition end instruction to the network camera 52 (S336). This end instruction is address-converted via the router 9 (S338), and the network camera 52
(S340).

When receiving the end instruction, the network camera 52 stops the imaging and stops the transmission of the moving image / sound data (S340).

As described above, according to the third embodiment, in addition to the same effects as those of the first embodiment, the user browses a moving image captured by the network camera in real time using a terminal device such as a personal computer. Will be able to.

(4) Other Embodiments In the second embodiment, the network camera 51 and the router 9 periodically transmit an IP address to the relay server 21, and the relay server 21 uses the received IP address to perform the network camera. A videophone connection between 51 and the mobile phone 34 is established. But,
If the router 9 is reset while establishing the connection, the global IP address may be changed. Therefore, it is desirable to provide the network camera 51 with life and death monitoring means having a function of constantly monitoring the operation of the router 9 and automatically detecting an abnormality of the router 9. For example, when the abnormality of the router 9 is detected during establishment of the TV phone connection, the alive monitoring means is configured to relay the relay server at a timing when the router 9 recovers and a new global IP address is assigned. 21. Camera information (camera ID, private IP
Address, port No). The router 9 converts the private IP address into the latest global IP address and transmits it to the relay server 21. When the relay server 21 receives the latest global IP address, the corresponding registered global IP is thereby received.
Update the address. As a result, even when the TV phone connection is interrupted by resetting the router 9 while the connection is established, the TV phone connection can be resumed based on the new global IP address. . It should be noted that the function of automatically detecting an abnormality of the router included in the alive monitoring means can be realized by using a conventional technique. The life and death monitoring means can also be applied to the network camera 52 in the third embodiment.

DESCRIPTION OF SYMBOLS 1 ... Image delivery system, 2 ... Relay server, 3 ... Terminal device, 4 ... LAN, 5 ... Imaging device, 6
Information processing terminal, 7 Information processing terminal, 8 Hub, 9 Router, 10 Recording / billing server
N ... Communication network

Claims (1)

An imaging device belonging to a local area network, a mobile phone terminal belonging to a network other than the local area network and having a videophone function, and a server configured to be able to communicate with each other,
First storage means for storing the identification information of the mobile phone terminal and the identification information of the imaging device in association with each other;
Second storage means for storing the identification information of the imaging device, the IP address, and the port number in association with each other;
An update unit that receives identification information, an IP address, and a port number of the imaging device that are periodically transmitted from the imaging device, and registers and updates corresponding information in the second storage unit;
A first extraction unit that extracts the identification information of the imaging device corresponding to the identification information of the mobile phone terminal from the first storage unit when receiving a videophone call from the mobile phone terminal;
Second extraction means for extracting from the second storage means an IP address and port number corresponding to the identification information of the imaging device extracted by the first extraction means;
Based on the IP address and port number extracted by the second extraction means, transfer means for transferring the outgoing videophone call to the imaging device;
Means for receiving video / audio data picked up by the image pickup device transmitted from the image pickup device in response to an outgoing videophone call transferred by the transfer means;
Means for transmitting the received video / audio data to the mobile phone terminal;
A server comprising:

Images