JP5542521B2 - Freely installable network camera device - Google Patents

Description

  The present invention relates to a network camera, and more specifically to a network camera capable of photographing a specified position within a specific range.

  The pan / tilt is designated from the controller to the network camera via the network, and the direction of the network camera is controlled.

  In such control via the network, there is a problem that the video displayed on the controller does not match the actual direction of the camera due to the delay time caused by the network.

  On the other hand, in Japanese Patent Application Laid-Open No. 2004-228620, the range that corresponds to the video area obtained from the current camera and the area that is shot after reflecting the pan and tilt transmitted to the camera are included in the camera's imageable area. By displaying the corresponding range, the camera can be easily directed to a desired position even when there is a delay time due to the network.

JP 2008-301191 A

  In general, an operator performs pan and tilt operations while viewing an image sent from a camera, and directs the camera to a desired position. In the pan / tilt operation, the camera changes its direction by a specified pan / tilt amount based on the current camera direction. For this reason, it is necessary for the operator to determine whether or not he or she is facing a desired position.

  If the fixed camera is directed in the direction in which the camera is a reference in advance, the desired position can be directed by giving the amount of pan and tilt by the difference from the desired direction.

  However, in a camera that is not fixed, the reference direction is not known in advance, and there is a problem in that it is not possible to know how much the pan and tilt amounts can be directed to the desired position.

  The present invention has been made in view of the above problems, and is suitable for a network camera capable of panning and tilting control even if the position of the camera and the direction of the camera are not known in advance. The main purpose is to provide a technique for estimating the direction of the camera and directing the camera to a desired position.

  The network camera of the present invention records an imaging unit that captures video, a network interface that outputs video captured by the imaging unit to a network, a control unit that controls the imaging unit according to instructions from a controller, and all-round video data And a position estimation unit that estimates a direction facing the position from the entire video data and the current video data.

  The controller of the present invention includes a user interface that displays an image from a network camera in response to an instruction input by an operator, a camera control unit that controls the network camera via the network interface in response to an instruction from the user interface, A recording unit that records all-round video data acquired from the network camera for all-round video acquisition, and a position that estimates the network camera position and the direction in which the network camera is facing from the all-round video data and the video data from the network camera An estimation unit and a network interface connected to the network are provided.

  The all-round video data acquisition method of the present invention is directed to an initial angle of the network camera, acquires video data from the network camera, records the current network camera angle and video data pair, and sets the current network camera angle. If the rotation unit is added, the process ends if the rotation angle is larger than the end angle. Otherwise, the network camera is rotated by the rotation unit angle, and the video data is repeatedly acquired from the network camera. It is characterized by acquiring video data.

  The position estimation method of the present invention is characterized in that a current position of a network camera and a direction in which the network camera is directed are obtained from all-round video data acquired in advance and current video data captured by the network camera. This is a position estimation method.

  The rotation amount determination method of the present invention is characterized in that the amount of rotation of the network camera is determined from the current position of the network camera, the direction in which the network camera is facing, and the position where the network camera is desired to be directed.

  The controller of the present invention includes a function of performing position estimation processing and rotation amount determination processing for a plurality of network cameras, and simultaneously directing the plurality of network cameras in a specific direction.

  The network camera of the present invention includes an imaging unit that captures an image, a network interface that outputs the image captured by the imaging unit to a network, and a control unit that controls the imaging unit according to an instruction from a controller. To do.

  The controller of the present invention has a function of instructing a plurality of network cameras to specify a position to be directed by multicast.

  The network camera of the present invention has a function of obtaining a necessary rotation amount by performing position estimation processing and rotation amount determination processing after a position is instructed from a controller, and facing a position instructed from the controller. .

  The position estimation method of the present invention is characterized in that accuracy is improved by using images of a plurality of network cameras.

  According to the present invention, for a network camera capable of controlling pan and tilt, even if the camera position and the direction in which the camera is facing are not known in advance, the camera position and the direction in which the camera is positioned are estimated, and the camera is desired. Can be directed to the position of

1 is a system configuration diagram of a network camera system 100 in the present embodiment. It is a block diagram of the controller 200 in this embodiment. It is a figure which shows the data structure of the perimeter video data 300 in this embodiment. It is a figure for demonstrating the processing flow of the perimeter video data acquisition process of the controller 200 in this embodiment. It is a figure which shows arrangement | positioning of the network camera in the perimeter video data acquisition process in this embodiment. It is a figure for demonstrating the position estimation process in this embodiment. It is a block diagram of the network camera 700 in this embodiment. It is a block diagram of the network camera 700 in this embodiment.

--- System configuration ---
Embodiments of the present invention will be described below in detail with reference to the drawings.

  FIG. 1 is a system configuration diagram of a network camera system 100 according to the present embodiment.

  A network camera system 100 shown in FIG. 1 includes a controller 200 that is connected to the network 101 and controls the network camera 700, a network camera 700 that is connected to the network 101 and controlled by the controller 200, and a network 101 that connects them. Configure.

--- Network camera ---
FIG. 7 is a configuration diagram of the network camera 700 in the present embodiment.

A network camera 700 illustrated in FIG. 7 includes an imaging unit 701 that captures a video, a network interface 210 that outputs the video captured by the imaging unit 710 to the network 101, and a control unit that controls the imaging unit 701 according to an instruction from the controller 200. 702.
The image capturing unit 701 has a function of capturing an image with an image sensor and a function of changing a direction of the network camera with a motor.
The control unit 702 receives an instruction from the controller 200 via the network interface 210 and controls an image sensor and a motor of the imaging unit 710.

---- Controller ---
FIG. 2 is a configuration diagram of the controller 200 in the present embodiment.

  The controller 200 shown in FIG. 2 receives an instruction input from the operator and controls the network camera 700 via the user interface 201 that displays an image from the network camera 700 and the network interface 210 that receives an instruction from the user interface 201. Position of the network camera 700 based on the camera control unit 202, the recording unit 203 that records the all-round video data 300 acquired from the all-round video acquisition network camera 700, and the video data from the all-round video data 300 and the network camera 700. And a position estimation unit 204 that estimates a direction in which the network camera 700 is facing, and a network interface 210 connected to the network 101.

---- All-round video data ---
FIG. 3 is a diagram showing a data structure of the all-round video data 300 in the present embodiment.

  The all-round video data 300 shown in FIG. 3 is data in which the angle and the video data are paired, and is acquired by the all-round video data acquisition process S400.

---- All-round video data acquisition process ---
FIG. 4 is a diagram for explaining the processing flow of the all-round video data acquisition processing of the controller 200 in the present embodiment.

  The controller 200 acquires an image of the entire circumference captured by the camera through an all-round video data acquisition process S400. Here, description will be made on the assumption that an image of the entire circumference is obtained using only horizontal rotation by panning, but the present invention includes a case of obtaining an image of the entire circumference by changing pan, tilt and zoom.

  The all-round video data 300 is data composed of video data acquired for each rotation angle from the initial angle to the end angle in the pan direction by the network camera 700 arranged in the center 501 of the target area.

  The procedure of the all-round video data acquisition process S400 will be described below.

  The position estimation unit 204 of the controller 200 extracts the rotation unit, the initial angle, and the end angle recorded by the position estimation unit 204 set in advance in the all-round video data acquisition process S400 (S401), and sets the network camera 700 to the initial state. The video data is acquired from the network camera 700 (S403) toward the angle (S403), and the current network camera 700 angle and the acquired video data are recorded in the recording unit 203 as a pair (S404). When the rotation unit is added to the angle of 700 (S405), the process ends when it becomes larger than the end angle. If the angle is not greater than the end angle, the network camera 700 is rotated by the angle of the rotation unit (S406), and then S403 to S405 are repeated.

---- Arrangement of network cameras in all-round video data acquisition processing ---
FIG. 5 is a diagram showing the arrangement of network cameras in the all-round video data acquisition process in the present embodiment.

  In the all-round video data acquisition processing, the network camera 700 is arranged in the center 501 of the target area in order to rotate the network camera 700 and photograph the entire target area 500.

  In FIG. 5, when the network camera 700 is arranged, the upper part of the figure is set to a reference angle of 0 degree.

--- Position estimation process ---
FIG. 6 is a diagram for explaining position estimation processing in the present embodiment.

In the position estimation process, the camera image and the all-round image are compared, and the direction θ in which the camera is facing, the distance d from the center 501 of the target area, and the horizontal position x are obtained.
First, in order to obtain θ, a camera image and an all-round image at an angle T (T = 0,5, ..., 355 degrees) are θ ′ (θ = −85, −80,..., 85 degrees) ) To obtain the T and θ ′ that are closest to the camera image.
When viewing the position O shown in the center of the all-round image at angle T from the camera V, the angle between the direction of the angle T and the direction reflected by the camera V is the angle θ ′, and the distance between the position O and the camera V is d ', If the horizontal position on the entire image is x' with O as the origin (x '= 0), the distance between V and x' is Sqrt (d '^ 2 + x' ^ 2- 2 * x '* d' * sinθ).
Considering perspective projection, each pixel P '(x', y ') in the camera image is each pixel P (x' * cosθ '/ Sqrt (d' ^ 2 + x '^ 2) -2 * x '* d' * sinθ), y '/ Sqrt (d' ^ 2 + x '^ 2-2 * x' * d '* sinθ)).
For each of θ ′ and d ′, the sum of squares D of the differences from the pixel values P of the perimeter video of the corresponding angle T is obtained for all the pixel values P ′ of the camera video.
The smaller this value D, the closer to the camera image the all-round image at angle T at θ ′, d ′.
Θ ′ that minimizes D is defined as a direction θ toward the camera.
Next, the horizontal position x of the camera is obtained.
A position x ″ that is closest to the all-round image at the angle T−θ when the camera image is moved x ″ in the horizontal direction with respect to θ ′ and d ′ that minimize D is obtained.
Each pixel P ′ (x ′, y ′) of the camera image moved x ″ in the horizontal direction is each pixel P ((x′-x ″) * cosθ ′ / Sqrt of the all-round image of angle T-θ. (d '^ 2 + (x'-x'') ^ 2-2 * (x'-x'') * d' * sinθ), y '/ Sqrt (d' ^ 2 + (x'-x '') ^ 2-2 * (x'-x'') * d' * sinθ)).

For each x ″, for each pixel value P ′ of the camera image, the square sum E of the difference from the corresponding pixel value P of the all-round image at the angle T is obtained.
The smaller the value E, the closer to the camera image the all-round image at the angle T-θ moved x ″ in the horizontal direction.

Let x ″ that minimizes E be the horizontal position x of the camera.
The distance from the center 501 of the target area to the position O ′ shown in the center of the all-round image at the angle T-θ is d ″, and d by d = d ″ -d′-x * sinθ is set as the target area. The distance d from the center 501 is assumed.

  As described above, the direction θ in which the camera is facing, the distance d from the center 501 of the target area, and the position x in the horizontal direction are obtained.

--- Rotation amount determination process ---
Next, the rotation amount determination process will be described.

  As described above, the obtained direction θ toward the camera, the distance d from the target area center 501, and the horizontal position x are set to the origin in the plane of the target area 500 shown in FIG. When converted to an XY Cartesian coordinate system with the reference angle direction as the X axis and the direction perpendicular to the X axis as the Y axis, the position of the network camera 700 is C (d * sinθ + x * cosθ, d * cosθ-x * sinθ). It becomes.

  If the position at which the network camera 700 is to be pointed is (X ′, Y ′) in the XY Cartesian coordinate system, the angle θ ″ for panning the network camera 700 is θ ″ = arctan ((X′−d * sinθ−x * cosθ) / (Y′−d * cosθ + x * sinθ)) − θ.

  By panning the network camera 700 by θ ″, the network camera 700 is directed to a desired position.

---- Summary ---
The controller 200 acquires the omnidirectional video data 300 by the omnidirectional video data acquisition process.

  The controller 200 obtains the direction facing the position of the network camera 700 from the all-round video data 300 acquired in advance and the video data acquired from the network camera 700 by position estimation processing.

  The controller 200 obtains an angle for panning the network camera 700 from the current position of the network camera 700, the direction in which the network camera 700 is directed, and the position where the network camera 700 is to be directed by the rotation amount determination process. By panning the network camera 700 by an angle, the network camera 700 is directed to a desired position.

  As described above, the controller 200 can point the network camera 700 to a desired position.

---- Multiple cameras ---
The controller 200 performs position estimation processing and rotation amount determination processing on each of the plurality of network cameras 700 in the target area 500, and directs the plurality of network cameras 700 to a specific position all at once.

  As described above, the controller 200 can direct a plurality of network cameras 700 to positions where the network cameras 700 are desired to be directed simultaneously.

--- The camera itself estimates the position ---
In the first embodiment, the controller 200 determines the position of the network camera 700. However, in the second embodiment, the network camera 700 records the all-round video data 300, the all-round video data acquisition process, and the position estimation process. A position estimation unit 204 that performs rotation amount determination processing, and the network camera 700 obtains the position of the network camera 700. Further, the network camera 700 itself can shoot the all-round video data 300. In addition, video data captured by the network camera 700 itself can be used as an input to the position estimation unit.

--- Network camera ---
FIG. 8 is a configuration diagram of the network camera 700 in the present embodiment.

  A network camera 700 illustrated in FIG. 8 includes an imaging unit 701 that captures a video, a network interface 210 that outputs the video captured by the imaging unit 710 to the network 101, and a control unit that controls the imaging unit 701 according to an instruction from the controller 200. 702, a user interface 201 for inputting an instruction from an operator, a recording unit 203 for recording all-round video data 300, and a direction from the all-round video data 300 and the current video data to the position of the network camera 700 And a position estimation unit 204 that estimates

  In this case, when the controller 200 indicates the position to be directed to the network camera 700, the network camera 700 obtains a necessary rotation amount by the position estimation process and the rotation amount determination process, and faces the instructed position.

  As described above, the controller 200 can point the network camera 700 to a desired position.

---- Multiple cameras ---
The controller 200 instructs a plurality of network cameras 700 in the target area 500 to be directed by multicast, so that the plurality of network cameras 700 are simultaneously directed to a specific position.

  As described above, the controller 200 can direct a plurality of network cameras 700 to positions where the network cameras 700 are desired to be directed simultaneously.

---- Improving the accuracy of position estimation using multiple cameras-
When the relative positional relationship between the plurality of network cameras 700 is known, the individual network cameras 700 do not calculate the sum of squares of the differences from the pixel values P of the all-round video image at the angle T. The network camera 700 is treated as one camera, and θ ′ and d ′ that minimize the sum of the square sum of the difference between the pixel values of the camera images of the network cameras 700 and the all-around video are obtained, thereby improving the accuracy of the position estimation process. Will improve.

  As mentioned above, although embodiment of this invention was described concretely based on the embodiment, this invention is not limited to these forms, In the range which does not deviate from the summary, various changes are possible.

DESCRIPTION OF SYMBOLS 100 Network camera system 101 Network 200 Controller 201 User interface 202 Camera control part 203 Recording part 204 Position estimation part 210 Network interface 300 All around video data 500 Target area 501 Target area center 701 Imaging part 702 Control part

Claims (4)

A network camera connected to a controller via a network,
An imaging unit for capturing images;
A network interface for outputting an image captured by the imaging unit to the network,
A control unit for controlling the image pickup unit according to an instruction from the controller,
A recording unit for recording the entire circumference image data and the angle of the network camera video data and the video data of the network camera is photographed is associated,
A position estimation unit for estimating a direction facing the position of the network camera based the on the entire circumference video data and current image data,
A network camera comprising: A controller connected to a network camera via a network,
A network interface connected to the network;
A user interface that displays video from the network camera receives an input of instruction by the operator,
A camera control unit that controls the network camera via the network interface receives an instruction from the user interface,
A recording unit that records video data acquired from the network camera and all-round video data in which the angle of the network camera at which the video data was captured is associated ;
A position estimation unit for estimating a direction facing the position of the network camera on the basis of the video data from the the entire circumference video data and the network camera,
Controller, characterized in that it comprises a. The controller according to claim 2, wherein
The control unit directs the network camera to the initial angle, it adds a predetermined rotational unit angle of Netw network camera, the image data obtained for each of the rotating unit by the network camera from the initial angle to the end angle A controller that is stored in the recording unit in association with an angle of the network camera. The controller according to claim 2, wherein
The control unit determines an amount to rotate the network camera from a direction facing the position of the network camera estimated by the position estimation unit and a position where the network camera is to be directed. Controller.

Images