JP7706872B2 - Information processing device, information processing method, and program - Google Patents

Description

The present invention relates to an information processing device, an information processing method, and a program.

In a conventional camera capable of automatic tracking, a technique is known in which, when the camera is reset, the pan/tilt mechanism is moved to an initial position, a subject for automatic tracking is detected, and the camera transitions to an automatic tracking state (for example, Patent Document 1).

Patent No. 6205070

The problem that this invention aims to solve is to shorten the time it takes to resume tracking when returning from standby mode to normal imaging mode.

In order to solve the above-mentioned problems, according to one aspect of the information processing device of the present invention, there is provided an information processing device having: a tracking means that executes a tracking process that changes the imaging direction of the imaging means based on the position of an object to be tracked detected from a video signal acquired by the imaging means; a storage means that outputs the video signal to the outside and stores a pan/tilt operating range of the imaging means by the tracking means in a first state in which the tracking process is effective ; and a switching means that switches between the first state and a second state in which the video signal is not output to the outside and the tracking process is not effective, wherein, when the switching means switches from the second state to the first state, the tracking means changes the imaging direction of the imaging means so as to image the pan/tilt operating range stored in the storage means before the second state is entered.

FIG. 1 is a block diagram showing an example of the configuration of a monitoring system according to a first embodiment. FIG. 1 is a block diagram showing an example of the functions of a monitoring system. 11 is a flowchart showing an example of processing in the monitoring system. FIG. 11 is a diagram showing an example of the operation of the monitoring system. FIG. 11 is a diagram showing an example of the operation of the monitoring system.

Below, an embodiment for carrying out the present invention will be described in detail with reference to the attached drawings. Note that the embodiment described below is one example of a means for realizing the present invention, and should be appropriately modified or changed depending on the configuration of the device to which the present invention is applied and various conditions, and the present invention is not necessarily limited to the embodiment below. Furthermore, not all of the combinations of features described in the embodiment are necessarily essential to the solution of the present invention.

(Embodiment 1)
As a first embodiment of the present invention, a surveillance system having a function of tracking a tracking target object (for example, a person) will be described below as an example.
FIG. 1 is a block diagram showing an example of the configuration of a monitoring system according to a first embodiment.
This surveillance system comprises a remote camera 1 with a pan/tilt mechanism (platform function), a controller 2 for controlling the direction of the remote camera 1, and a display 3 for displaying images and the like being captured.

FIG. 2 is a block diagram showing an example of the functionality of the monitoring system of FIG.
The remote camera 1 includes an imaging unit 11 that captures an image of a subject, an image processing unit 12 that processes the image captured by the imaging unit 11, a system control unit 13 that controls the entire device, and a storage unit 14 that stores data such as setting values. The remote camera 1 also includes a pan driving unit 15 and a tilt driving unit 16 that change the imaging direction of the imaging unit 11, a speed control unit 17 that controls the speed at which the imaging direction is changed, and a subject analysis unit 18 that analyzes the captured subject. The subject analysis unit 18 includes a movement amount calculation unit 19 that calculates the amount of movement of the subject, and a subject identification unit 20 that identifies the subject. The remote camera 1 also includes a tracking unit 21 that tracks the tracking target, a tracking target search unit 22 that searches for the tracking target subject, a communication unit 23 that communicates with the controller 2, and a standby mode control unit 24 that controls the operation mode. The remote camera 1 has two operation modes: a normal imaging mode in which the captured image is output, and a standby mode in which the imaging direction of the imaging unit 11 is changed to a predetermined position (direction) without outputting the captured image.

The imaging unit 11 is equipped with a lens and an imaging element, and captures an image of a subject and converts it into an electrical signal. A zoom function may be provided to perform a zoom operation to change the angle of view of the imaging unit 11. In this case, the zoom operation may be performed under control of the speed control unit 17 or the system control unit 13. The image processing unit 12 performs image processing, compression encoding processing, etc. on the imaging signal converted into an electrical signal by the imaging unit 11 to generate image data, which is then supplied to the system control unit 13. The storage unit 14 stores image quality adjustment parameters, network setting values, etc. This makes it possible to refer to previously set values even when the remote camera 1 is restarted. In this embodiment, the storage unit 14 also stores the range in which panning and tilting operations were performed in normal imaging mode.

The pan drive unit 15 is equipped with a mechanical drive system (pan drive mechanism) that performs panning operations and a motor that serves as its drive source; by driving the motor, the pan drive mechanism is driven in the pan direction. The tilt drive unit 16 is equipped with a mechanical drive system (tilt drive mechanism) that performs tilting operations and a motor that serves as its drive source; by driving the motor, the tilt drive mechanism is driven in the tilt direction. The speed control unit 17 calculates the next drive speed to be instructed based on the current speed, target speed, acceleration value, or acceleration/deceleration table information, and instructs the pan drive unit 15 and tilt drive unit 16 to drive at the calculated speeds.

The subject analysis unit 18 analyzes the type and amount of movement (movement amount) of the subject in the captured image. The movement amount calculation unit 19 calculates the speed of the subject in the captured image from the trajectory of the subject's coordinates on the captured image. The movement amount calculation unit 19 also calculates the drive speed required to follow the subject (hereinafter referred to as the following speed) from the current imaging direction and the speed of the subject in the captured image. The subject identification unit 20 identifies the subject (object) using known techniques such as face detection and human body detection. In this embodiment, the subject is a person. Note that since techniques such as face detection and human body detection are known, detailed explanations will be omitted.

The tracking unit 21 transmits instructions to the pan driving unit 15 and the tilt driving unit 16 to perform pan/tilt operations so that the subject (object) designated as the tracking target is included in the captured image captured by the imaging unit 11. When transitioning from standby mode to normal imaging mode, the tracking target search unit 22 starts searching for the tracking target from the pan/tilt operation range that was being tracked before transitioning to standby mode. The communication unit 23 executes processing of network communication via a LAN (Local Area Network) or the like.

The system control unit 13 distributes image data generated by the image processing unit 12 to the controller 2 via the communication unit 23. The system control unit 13 also analyzes control commands supplied from the controller 2 via the communication unit 23 and executes processing according to the command. Specifically, the system control unit 13 executes processing such as instructions for setting imaging parameters, such as setting image quality adjustments for the image processing unit 12 and setting pan/tilt operations for the pan drive unit 15 and tilt drive unit 16.

The standby mode control unit 24 executes switching control between the normal imaging mode and the standby mode. In the normal imaging mode, the standby mode control unit 24 enables pan driving, tilt driving, imaging, video distribution, and automatic tracking. In the standby mode, the standby mode control unit 24 stops imaging, video distribution, and automatic tracking, and changes the imaging direction of the imaging unit 11 to a specified position (e.g., rearward) using pan/tilt operations, and then stops the pan/tilt operations.

The controller 2 includes an input unit 31 for inputting operations from the user, a system control unit 32 for controlling the entire device, and a communication unit 33 for communicating with the remote camera 1. The input unit 31 includes buttons, a joystick, and the like, and inputs various operations from the user. The system control unit 32 transmits control commands to the remote camera 1 via the communication unit 33 in response to operations from the user. The display 3 also includes a display unit 41 having a display screen. The display unit 41 displays, for example, the UI (User Interface) screen of the controller 2, and the control results of the remote camera 1 received via the communication units 23 and 33. The display unit 41 also displays the video delivered from the remote camera 1. In this embodiment, the remote camera 1 is provided with a subject analysis unit 18, which performs processing to analyze the type and amount of movement (amount of movement) of the subject in the captured image, but this configuration is not limited to this. For example, the remote camera 1 may be configured to transmit the captured image via the communication unit 23 to a server device or the like connected to a network, and the server device or the like may execute processing equivalent to that of the subject analysis unit 18. In addition, the configuration may be changed as appropriate, such as making the imaging unit 11 and the other components separate devices.

An example of processing in the monitoring system of this embodiment will be described below with reference to FIG.
In the following description, it is assumed that the startup of the remote camera 1 and the controller 2 has already been completed, the controller 2 is in a state where it can receive input operations from the user, and the subject analysis unit 18 is in a state where it can perform image analysis on the image captured by the remote camera 1.

First, in step S1, the controller 2 inputs information indicating a tracking target based on an instruction from a user via the input unit 31. Specifically, for example, an image of a person recognized by the subject identification unit 20 may be displayed on the display unit 41, and the user may select a tracking target from the displayed image of the person. For example, as shown in FIG. 4, in a state in which the remote camera 1 captures images of three subjects, the subject analysis unit 18 detects three subjects (persons) and assigns IDs (001 to 003) to the detected people. The system control unit 13 supplies images of the people to which IDs have been assigned to the controller 2, and the system control unit 32 causes the display unit 41 to display the supplied images of the people. When the user selects one of the displayed images of the people, the system control unit 32 transmits the ID of the selected person to the remote camera 1. In the following description, it is assumed that the user selects person 001 as the tracking target.

In step S2, the remote camera 1 transitions to a normal imaging mode, and the tracking unit 21 starts tracking (automatic tracking) the object to be tracked that has been input via the input unit 31. Note that in this embodiment, in the normal operation mode, the speed control unit 17 causes the pan driving unit 15 and the tilt driving unit 16 to perform pan/tilt operations so that the object to be tracked is positioned at the center of the captured image captured by the imaging unit 11 of the remote camera 1.
In step S3, the storage unit 14 stores the range of pan/tilt movement (tracking range) driven during tracking. For example, in the example of FIG. 4 described above, assume that the person 001 to be tracked pans in the range of -45 deg to 45 deg during tracking (note that in the example of FIG. 4, only the pan direction is shown). Here, the pan angle when the remote camera 1 faces horizontally in the positive direction of the Y axis is set to 0 deg, and the X axis (right) direction is set to a positive angle. Also, the tilt angle when the remote camera 1 faces horizontally in the positive direction of the Y axis is set to 0 deg, and the Z axis (up) direction is set to a positive angle.

In step S4, the standby mode control unit 24 determines whether or not a standby mode instruction command (standby instruction) has been input from the input unit 31. If the standby mode instruction command has not been input, the remote camera 1 returns to step S3 and repeats the process of the normal imaging mode. If the standby mode instruction command has been input, the process proceeds to step S5, where the system control unit 13 transmits an imaging stop instruction command to the imaging unit 11 and the image processing unit 12, and stops the video output to the display unit 41. In addition, the imaging unit 11 transmits a tracking stop instruction command to the subject analysis unit 18 and the tracking unit 21, and stops tracking.
In step S6, the pan driving unit 15 and the tilt driving unit 16 perform pan and tilt operations to a predetermined position (for example, a specified rear position) so that the remote camera 1 faces backward (tilt angle 0 deg, pan angle -180 deg) as shown in Fig. 5. This causes the remote camera 1 to transition to standby mode.

In step S7, the standby mode control unit 24 determines whether or not a normal imaging mode instruction command (return instruction) has been input from the input unit 31. If a normal imaging mode instruction command has not been input, the standby mode control unit 24 repeats the process of step S7. This allows the operation in the standby mode to continue.
If a normal imaging mode instruction command is input, the process proceeds to step S8, where the tracking target search unit 22 determines whether or not the resumption of automatic tracking has been instructed. It should be noted that, for example, it is possible to set in advance whether the resumption of automatic tracking is enabled or disabled by a user's input operation. If the resumption of automatic tracking has not been instructed, the process proceeds to step S13, where the pan driving unit 15 and the tilt driving unit 16 perform pan/tilt operations to move the orientation of the remote camera 1 to the home position, and the process ends. For example, the home position is a predetermined initial position set in the remote camera 1, such as a position where the pan angle is 0 deg and the tilt angle is 0 deg.

If an instruction to resume automatic tracking is given, the process proceeds to step S9, where the pan driver 15 and the tilt driver 16 perform pan/tilt operation to the end position (tracking target search start position) of the pan/tilt operation range tracked before standby. For example, as shown in Fig. 5, pan operation is performed to the tracking target search start position at a pan angle of -45 deg (note that the example in Fig. 5 shows operation only in the pan direction).
In step S10, the system control unit 13 transmits an image output instruction command to the image capturing unit 11 and the image processing unit 12, and restarts image output to the display unit 41.

In step S11, the pan driving unit 15 and the tilt driving unit 16 continue the pan/tilt operation toward the end of the other tracking range (the tracking target search end position) (in the example of FIG. 5, the operation is shown as being in the pan direction only).
In step S12, the tracking target search unit 22 judges whether or not the tracking target is within the pan/tilt operation range tracked before transitioning to standby mode (before standby). If it is not within the range, that is, if the tracking target cannot be detected between the tracking target search start position and the tracking target search end position, the direction of the remote camera 1 is moved to the home position (initial position) in step S13, and the process ends. If it is within the range, the process proceeds to step S14, where the tracking target search unit 22 searches for the tracking target and judges whether or not the tracking target has been detected. If the tracking target has not been detected, the process returns to step S11, and the pan/tilt operation is continued.

If a tracking target is detected, the process proceeds to step S15, where the tracking unit 21 resumes tracking of the tracking target. This returns the camera from standby mode to normal imaging mode, and the process returns to step S3 to continue. For example, if person 001 has been selected as the tracking target in advance by a user input operation, person 001 is detected as the tracking target in the process of step S14, and after detection, the tracking unit 21 resumes tracking person 001 in step S15.
The start condition of the process in FIG. 3 may be based on a user instruction, or may be determined and started automatically by the remote camera 1 after the remote camera 1 is started.

By the above process, when the mode is switched to the standby mode during tracking (normal imaging mode) and then switched to the normal imaging mode, it is possible to start searching for the tracking target from the pan/tilt operation range during tracking before the transition to the standby mode. The range in which the pan/tilt operation was not performed during tracking before the transition to the standby mode is considered to be a range that does not need to be imaged. Therefore, in this embodiment, when the mode is switched to the normal imaging mode, it is possible to image an appropriate range as required. This allows the tracking function to be restored in a short time.
Also, in some cases, capturing an image of a range that does not need to be captured may be undesirable from the viewpoint of privacy protection, for example, in a concert hall, a lecture recording, a lecture, etc. In this embodiment, the above-described process makes it possible to restore the tracking function in a short time while avoiding capturing an image of a range that does not need to be captured from the viewpoint of privacy protection.

In this embodiment, the tracking target is specified in response to an operation by the user, but the remote camera 1 may automatically determine the tracking target.
In the present embodiment, the tracking target search unit 22 searches for the tracking target by performing only pan/tilt operations, but the present invention is not limited to this. For example, the tracking target search unit 22 may search for the tracking target after performing a zoom operation so that the imaging area is within the tilt operation range stored before the transition to the standby mode. This allows the tracking target to be searched for in an appropriate imaging area while minimizing pan/tilt operations, thereby improving the efficiency of the search.

In addition, in this embodiment, if the tracking target search unit 22 does not detect a tracking target within the pan/tilt operation range during tracking before transition to standby mode, the tracking target search unit 22 returns to the home position in step S13, but this is not limited to this. For example, in step S13, the tracking target search unit 22 may perform a search outside the pan/tilt operation range during tracking before transition to standby mode (normal shooting mode) by using pan/tilt/zoom operations in combination. For example, by expanding the imaging range by zooming, it is possible to minimize pan/tilt operations and increase the possibility of detecting the tracking target compared to a case where a search outside the pan/tilt operation range is performed by pan/tilt operations alone.

(Modification)
Although the preferred embodiments of the present invention have been described above, the present invention is not limited to these embodiments, and various modifications and changes are possible within the scope of the gist of the present invention. In addition, the present invention can be embodied as, for example, a system, an apparatus, a method, a program, or a recording medium (storage medium).

The present invention can also be realized by supplying a program that realizes one or more of the functions of the above-mentioned embodiments to a system or device via a network or storage medium, and having one or more processors in the computer of the system or device read and execute the program. It can also be realized by a circuit (e.g., an ASIC) that realizes one or more functions. ASIC is an abbreviation for Application Specific Integrated Circuit.

1...Remote camera, 2...Controller, 3...Display, 11...Imaging unit, 12...Image processing unit, 13...System control unit, 14...Storage unit, 15...Pan driving unit, 16...Tilt driving unit, 17...Speed control unit, 18...Subject analysis unit, 19...Movement amount calculation unit, 20...Subject identification unit, 21...Tracking unit, 22...Tracking target search unit, 23...Communication unit, 24...Standby mode control unit, 31...Input unit, 32...System control unit, 33...Communication unit, 41...Display unit

Claims (11)

a tracking means for executing a tracking process for changing an imaging direction of the imaging means based on a position of an object to be tracked detected from a video signal acquired by the imaging means;
a storage means for storing a pan/tilt operation range of the imaging means by the tracking means in a first state in which the tracking process is enabled, and outputting the video signal to an external device;
a switching means for switching between the first state and a second state in which the video signal is not output to the outside and the tracking process is not enabled;
having
When the switching means switches from the second state to the first state,
the tracking means changes the imaging direction of the imaging means so as to image the pan/tilt operation range stored in the storage means before the second state is reached;
23. An information processing apparatus comprising: a search means for detecting a position of the object to be tracked in response to the video signal after the imaging direction of the imaging means is changed by the tracking means;
The information processing apparatus according to claim 1 , further comprising: The tracking means includes:
when the position of the object to be tracked is detected by the searching means, control of the imaging means is resumed.
3. The information processing apparatus according to claim 2. The searching means includes:
When detecting the position of the object to be tracked, a zoom device is controlled to change the angle of view of the imaging means based on the pan/tilt operation range stored in the storage means.
4. The information processing apparatus according to claim 2, wherein the information processing apparatus is a computer. When the position of the object to be tracked is not detected by the search means, the tracking means changes the imaging direction of the imaging means to a predetermined initial position.
5. The information processing device according to claim 2, wherein the information processing device is a computer. The searching means includes:
When the tracking target is not detected within the pan/tilt operation range stored in the storage means,
Detecting the tracking target outside the pan/tilt operation range.
6. The information processing device according to claim 2, wherein the information processing device is a computer. acquiring an image signal of the captured image and detecting a position of an object to be tracked;
executing a tracking process for changing an imaging direction based on a position of the object to be tracked;
outputting the video signal to an external device and storing a pan/tilt operation range of an imaging means in a first state in which the tracking process is enabled;
switching between the first state and a second state in which the video signal is not output and the tracking process is not enabled;
changing an imaging direction when switching from the second state to the first state so as to image the pan/tilt operation range stored before the second state is entered;
13. An information processing method comprising: A program for causing a computer to function as each of the means of an information processing device according to any one of claims 1 to 6. a tracking means for executing a tracking process for changing an imaging direction of the imaging means based on a position of a tracking target detected from a video signal acquired by the imaging means;
a storage means for storing a pan/tilt operation range of the imaging means by the tracking means in a first state in which the tracking process is enabled;
A switching means for switching between the first state and a second state in which the tracking process is not enabled;
having
When the switching means switches from the second state to the first state,
the tracking means changes the imaging direction of the imaging means so as to image the pan/tilt operation range stored in the storage means before the second state is reached;
23. An information processing apparatus comprising: a tracking process for performing a tracking process of changing an imaging direction of the imaging means based on a position of a tracking target detected from a video signal acquired by the imaging means;
a storage step of storing a pan/tilt operation range of the imaging means in a first state in which the tracking process is enabled;
a switching step of switching between the first state and a second state in which the tracking process is not enabled;
having
When switching from the second state to the first state in the switching step,
changing an imaging direction so as to image the pan/tilt operation range stored before the second state is entered;
23. An information processing method comprising: A program for causing a computer to function as each of the means of the information processing device described in claim 9.

Images