JP4855797B2 - Surveillance system and image processing apparatus - Google Patents

Description

  The present invention relates to a monitoring system and an image processing apparatus suitable for performing various types of monitoring such as road monitoring and intrusion monitoring using a camera.

  In the conventional monitoring system, as shown in FIG. 22, analog video signals to be monitored captured by the monitoring cameras 101-1, 101-2,... Are directly transmitted to the event detection apparatus 102. It was. In order to transmit an analog video signal, it is necessary to provide an equalizer 103 in the transmission line that suppresses the deterioration. The event detection apparatus 102 captures a video signal, performs A / D conversion, performs image processing, and detects an event. A plurality of videos obtained by the monitoring cameras 101-1, 101-2,... When a single event detection device 102 processes the video signal, the video distributor 104 switches the analog video signal and sends it to the event detection device 102.

  In this system, instantaneous switching by the video distributor 104 is impossible, and control for switching the event detection device 102 by the video distributor 104 cannot be performed, and videos taken by a required monitoring camera one after another. However, image processing by time division cannot be executed. The recorder 105 records an analog video signal. The recorder 105 is configured to select and record a necessary analog video signal under the control of the event detection device 102. The video signal recorded in the past will be recorded later. It was impossible to read and perform image analysis.

On the other hand, Patent Document 1 discloses that video signals obtained by a plurality of monitoring cameras are transmitted by MPEG, stored together with date information, and a plurality of images are displayed based on the date information at the time of reproduction. Further, Patent Document 2 discloses a monitoring system that sends only an I frame in a normal state and sends an I frame, a B frame, and a P frame when a predetermined motion is detected.
JP 2003-163923 A JP 2000-165847 A

  However, when image processing is performed on images captured by a plurality of cameras, there is no system constructed from the viewpoint of efficient analysis at one or a plurality of monitoring terminals, and images are captured by a plurality of cameras. Appearance of a monitoring system and an image processing apparatus suitable for performing image processing on a video is required.

  The problem to be solved by the present invention is that there is no system suitable for performing image processing on images captured by a plurality of cameras, and an object of the present invention is to perform image processing on images captured by a plurality of cameras. It is an object of the present invention to provide a monitoring system and an image processing apparatus that are suitable for performing.

A monitoring system according to the present invention includes a camera that captures a subject to be imaged, a plurality of video transmission units that output encoded video information that can be transmitted over a network with respect to the captured video, and a network to which the plurality of video transmission units are connected. And at least I frames included in the video information transmitted from the plurality of video transmission units connected to the network and, if necessary, B frames and P frames are captured and converted into digital image information for one screen. A decoder;
Image analysis is performed in parallel processing in which a plurality of image processings for detecting an object is performed in parallel on at least one I-frame image which is digital image information corresponding to a plurality of video transmission units converted by the decoder, and event occurrence and disappearance The image processing means for detecting the digital image information corresponding to one video transmission unit related to parallel processing in the event of an event transfer to another image processing means, and the I, B, P frames are And image processing means for performing the processing used and returning the processing to the original parallel processing when the event disappears .

In the monitoring system according to the present invention, the image processing means has a plurality of tasks, change the assignment proportion of CPU resources to each task related to digital image information according to the transmission of the corresponding picture sending unit in accordance with the event occurrence annihilation It is characterized by doing.

  The monitoring system according to the present invention is characterized in that the image processing means is provided in one apparatus.

  In the monitoring system according to the present invention, the image processing means is realized by a plurality of devices.

  In the monitoring system according to the present invention, the image processing means notifies the image processing means that processes the digital image information of the corresponding camera according to the analysis result.

  In the monitoring system according to the present invention, the camera includes a zoom mechanism and a turning mechanism, and includes a camera control means for controlling the zoom mechanism and the turning mechanism in the corresponding camera according to the analysis result by the image processing means. It is characterized by being.

  The monitoring system according to the present invention includes a patrol data table in which turning speed information and zoom information are associated with turning angle information, and the camera control means is a zoom mechanism in a corresponding camera based on information in the patrol data table. The swivel mechanism is controlled.

  In the monitoring system according to the present invention, the camera control means receives the event occurrence information sent via the network, and controls the zoom mechanism and the turning mechanism in the corresponding camera according to the event occurrence information. .

  In the monitoring system according to the present invention, the image processing means notifies the camera control means provided in each of the plurality of apparatuses according to the analysis result, while the camera control means in the apparatus that has received the notification responds to the notification. The zoom mechanism and the turning mechanism of the corresponding camera are controlled.

  The surveillance system according to the present invention is characterized in that it is provided with modified video creation means for creating a panoramic video based on video information from several cameras.

  The monitoring system according to the present invention is characterized in that it is provided with processed video creation means for creating a bird's-eye view video showing the movement status of the moving body based on video information from several cameras.

  In the monitoring system according to the present invention, video storage means for capturing and storing video information transmitted from the video transmission section from the network, and information for acquiring video before and after the occurrence of the event from the video storage means as image information at the time of the event occurrence And an event occurrence notification image providing means for sending an event occurrence notification image set to the client to the client.

The image processing apparatus according to the present invention, a decoder for converting optionally write Mutotomoni takes at least I-frames included in the video information to the B-frame P-frame also takes Write-one screen digital image information, the Image processing means for performing image analysis by parallel processing in which a plurality of image processes for detecting an object are performed in parallel on at least one I-frame image that is digital image information converted by a decoder, and detecting occurrence and disappearance of events. When an event occurs, the digital image information processing corresponding to one video transmission unit related to parallel processing is transferred to another image processing means, and processing using I, B, and P frames is performed. And image processing means for returning the processing to the original parallel processing .

  In the monitoring system according to the present invention, the plurality of video transmission units include a camera that captures an object to be imaged, and a plurality of video transmission units that output encoded video information that can be transmitted over the network with respect to the captured video via the network. At least I frame included in the transmitted video information is captured and converted into digital image information for one screen, and the digital image information corresponding to the plurality of converted video transmission units is analyzed by parallel processing, and event detection is performed. Therefore, there is an effect that it is possible to perform appropriate image processing with respect to videos captured by a plurality of cameras. In the image processing apparatus according to the present invention, at least an I frame included in video information is captured and converted into digital image information for one screen, the converted digital image information is subjected to image analysis by parallel processing, and event detection is performed. Therefore, there is an effect that appropriate image processing can be performed on the video imaged by the camera.

  In the monitoring system according to the present invention, the CPU resource allocation ratio relating to the digital image information related to the transmission of the corresponding video transmission unit is changed according to the analysis result, and the corresponding digital image information of the camera is determined according to the analysis result. Since notification is made to the image processing means to be processed, appropriate parallel processing proceeds according to the analysis result.

  The monitoring system according to the present invention captures at least an I frame included in video information, converts it into digital image information for one screen, and converts the converted digital image information corresponding to a plurality of video transmission units into an image by parallel processing. By adopting a configuration for analyzing and detecting an event, the object of performing appropriate image processing on images captured by a plurality of cameras is achieved.

  Embodiments of a monitoring system and an image processing apparatus according to the present invention will be described below with reference to the accompanying drawings. In each figure, the same components are denoted by the same reference numerals, and redundant description is omitted. FIG. 1 shows an embodiment of a monitoring system according to the present invention. In this monitoring system, an IP camera 11-1 is a plurality of video transmission units that include a camera that captures an object to be imaged and that outputs encoded video information (MPEG2, MPEG4 video information, etc.) that can be transmitted over the network for the captured video. 11-2 and 11-3 are connected to the network 10. The video transmission unit is not limited to the IP camera. As shown in FIG. 2, the analog camera 12 that outputs a video signal such as an NTSC signal and the encoded video information (MPEG2, MPEG4) are encoded. It is also possible to use an encoder 13 that converts the video information into the video information and the like. The number of video transmission units is not limited to three as long as it is plural.

  A monitoring terminal 20 constituted by a personal computer is connected to the network 10. The monitoring terminal 20 is connected with an image display unit 30 by a display of a personal computer, and is provided with a decoder 21, an image processing unit 22, and a display control unit 23. The decoder 21 captures at least an I frame included in the video information transmitted from the IP cameras 11-1, 11-2, and 11-3, which are the plurality of video transmission units, and outputs a digital image for one screen for image processing. It is converted into information. The image processing means 22 analyzes the digital image information corresponding to the plurality of IP cameras 11-1, 11-2, 11-3 decoded by the decoder 21 through parallel processing, and detects an event.

  The display control means 23 uses the image analysis and event detection results by the image processing means 22 to perform display based on the decoded image information for the video information sent by the IP cameras 11-1, 11-2, 11-3. It is.

  In the monitoring terminal 20 constituted by a personal computer, the CPU functions as the decoder 21, the image processing means 22, and the display control means 23 by performing an operation based on a program corresponding to the flowchart shown in FIG. The operation will be described based on this flowchart. At the start, video information sent by the three IP cameras 11-1, 11-2, and 11-3 is captured and decoded, and image processing for these is performed in an equivalent parallel process (S11) to detect the occurrence of an event. Perform (S12). The video information in the above includes an I frame, a B frame, and a P frame. Here, only the I frame is captured and decoded into luminance information corresponding to each pixel. For this reason, since I frames reach a maximum of two frames per second, only the I frames are captured for the three IP cameras 11-1, 11-2, and 11-3. The image processing by the image processing means 22 uses an image processing algorithm that can detect an object even in intermittent video, for example, an edge extraction process.

  In the detection of the event occurrence in step S12, if no event has occurred, the process returns to step S11 to continue the processing. On the other hand, if the event occurrence is confirmed in step S12, the image information of the decoding result corresponding to one camera in which the event has occurred is displayed. Many resources are allocated to perform image processing (S13). For example, as shown in FIG. 4, the video information decoding and image processing software include a task T1 for video corresponding to the IP camera 11-1, a task T2 for video corresponding to the IP camera 11-2, and an IP. It is independent of the task T3 for video corresponding to the camera 11-3, uses CPU resources, and the task manager TM performs processing by assigning many resources to any of the tasks T1 to T3. To control. As a result, in step S11, resources are allocated evenly to tasks T1 to T3 as shown in FIG. 5A at the beginning, whereas for example, as shown in FIG. Allocate resources and process. In a task with a lot of resources allocated, not only I frame but also P frame or B frame can be decoded, and monochrome image to color image can be decoded. Processing accuracy is improved by performing image processing by changing algorithms or combining several algorithms. For example, the image display unit 30 displays a decoded image of the camera video of one camera related to the occurrence of an event (display control unit 23).

  As to the result of the processing in step S13, it is detected whether the event that has occurred has disappeared (S14). If the event has disappeared, the process returns to step S11, and image processing is performed by equal parallel processing. If the event does not disappear as a result of the determination in step S14, it is further detected whether an event has occurred in the processing result for the image information of the decoding results corresponding to the remaining two cameras (S15), and the event is detected in step S15. If the occurrence cannot be detected, the process returns to step S13 to continue the process. On the other hand, if the occurrence of an event is detected in step S15, a large amount of image information in the decoding result corresponding to the two cameras in which the event has occurred at this time is stored. Image processing is performed by allocating resources (S16). For example, as shown in FIG. 5C, the processing of the task T3 in which no event occurs is stopped, and the processing is performed by allocating resources only to the tasks T1 and T2. The image display unit 30 displays, for example, an image decoded for the camera images of the two cameras related to the occurrence of the event by dividing the screen into two (display control means 23).

  Following step S16, it is detected whether the event that has occurred has disappeared (S17). If the event has disappeared, the process returns to step S13. If the result of determination in step S17 is that the event has not disappeared, the process proceeds to step S16. Continue processing. Through such processing, resource allocation is performed according to the occurrence and disappearance of an event, and image processing on images captured by a plurality of cameras is appropriately performed.

  FIG. 6 shows a second embodiment of the monitoring system according to the present invention. In this monitoring system, three monitoring side terminals 20-1, 20-2 and 20-3 are connected to the network 10. The monitoring terminal 20-1 includes an image display unit 30-1, the monitoring terminal 20-2 includes an image display unit 30-2, and the monitoring terminal 20-3 includes an image display unit 30-3. Is provided. Reference numerals 21-1, 21-2, and 21-3 denote decoders, 22-1, 22-2, and 22-3 denote image processing means, and 23-1, 23-2, and 23-3 denote display control means. As shown in FIG. 7, the monitoring terminal 20-1 has processing request information in which two processing request destination addresses (addresses corresponding to the monitoring terminals 20-2 and 20-3) and a busy flag are set. Is provided.

  The CPU of the monitoring terminal 20-1 performs processing corresponding to the flowchart shown in FIG. 8, whereas the CPUs of the monitoring terminals 20-2 and 20-3 receive processing requests from the monitoring terminal 20-1. The image processing corresponding to the flowchart shown in FIG. The display control means 23-1, 23-2, 23-3 display on the image display units 30-1, 30-2, 30-3 an image obtained by decoding the camera image of the corresponding camera that is processing the image. Is displayed (display control means 23). The operation of this system will be described below with reference to the flowcharts shown in FIGS. The CPU of the monitoring terminal 20-1 is started, and the video information sent from the three IP cameras 11-1, 11-2, 11-3 is fetched and decoded, and image processing for these is performed by equal parallel processing. (S11), event occurrence is detected (S12).

  In the event occurrence detection in step S12, if no event has occurred, the process returns to step S11 and continues the process. On the other hand, if the event occurrence is confirmed in step S12, the process request information shown in FIG. Search for a monitoring terminal (with no flag set), request this monitoring terminal (for example, 30-2) to perform image processing of the camera video corresponding to the camera related to the event occurrence, set the flag, and generate the event The camera video corresponding to the two cameras not yet decoded is decoded and image processing is performed (S23). As a result, the CPU of the monitoring terminal 20-1 performs image processing on the camera video corresponding to the two cameras, so that a large amount of resources can be allocated for one camera, and not only the I frame but also the P frame, Processing accuracy is improved by performing decoding using a B frame, decoding from a monochrome image to color image, changing a processing algorithm, or performing image processing by combining several algorithms. For example, the image display unit 30-1 displays a decoded image of the camera images of two cameras in which no event has occurred (display control means 23).

  Following the processing in step S23, it is detected whether a notification that the event has disappeared has arrived from the monitoring-side terminal that is the processing request destination (S24). If there is an event disappearance notification, the process returns to step S11 to perform image processing. Are performed by equal parallel processing. As a result of the determination in step S24, if there is no notification of event disappearance, it is further detected whether an event has occurred in the processing result of the decoded image information corresponding to the remaining two cameras (S25), and in step S25 If the occurrence of an event cannot be detected, the process returns to step S23 to continue the processing. On the other hand, if the occurrence of an event is detected in step S25, the next (flag is not set) with reference to the processing request information shown in FIG. ) Search for the monitoring terminal, request the monitoring terminal (for example, 30-3) to perform image processing of the camera video corresponding to the camera related to the event occurrence, set the flag, and one camera in which no event has occurred The camera image corresponding to is decoded and image processing is performed (S26). At this time, on the image display unit 30-1, for example, an image decoded for the camera video of one camera in which no event has occurred is displayed.

  Following step S26, it is detected whether or not a notification indicating that the event has disappeared has arrived from the monitoring terminal that is the processing request destination (S27). If there is an event disappearance notification, the process returns to step S23, and the determination in step S27 is made. As a result, if there is no notification of event disappearance, even if an event occurrence is detected for the image processing handled by the monitoring terminal 30-1, the processing in step S26 is continued. By such processing, load distribution is performed according to the occurrence and disappearance of an event, and image processing on images captured by a plurality of cameras is appropriately performed.

  On the other hand, in response to a processing request from the monitoring terminal 20-1, the CPU of the monitoring terminals 20-2 and 20-3 starts a program corresponding to the flowchart shown in FIG. The video information sent from the camera is captured and decoded, image processing is performed on these (S31), and the disappearance of the event is detected (S32).

  If event disappearance cannot be detected in step S32, the process returns to step S31 and continues. At this time, the image display unit 30-2 or the image display unit 30-3 displays a decoded image of the camera video of one camera related to the processing request. If event disappearance is detected in step S32, an event disappearance notification is sent to the monitoring terminal 20-1 associated with the processing request source (S33), and the process ends.

  In the monitoring system according to the third embodiment, the configuration of the system is as shown in FIG. 6, but the IP cameras 11-1, 11-2, 11-3 have adjacent fields of view. For example, the moving body captured by the IP camera 11-1 moves from the field of view of the IP camera 11-1 to the field of view of the IP camera 11-2, and further moves from the field of view of the IP camera 11-2 to the field of view of the IP camera 11-3. Is placed in a different situation. Further, the CPUs of the monitoring side terminals 20-1, 20-2, 20-3 decode the video information sent by the IP cameras 11-1, 11-2, 11-3 assigned to them, respectively, and Since the processing corresponding to the flowchart shown in FIG. 10 is performed, the operation of this system will be described below with reference to the flowchart shown in FIG.

  Here, the moving body captured by the IP camera 11-1 moves from the field of view of the IP camera 11-1 to the field of view of the IP camera 11-2, and further from the field of view of the IP camera 11-2 to the field of view of the IP camera 11-3. The monitoring side terminal 20-1 holds the address of the monitoring side terminal 20-2 whose view is adjacent, and the monitoring side terminal 20-2 holds the address of the monitoring side terminal 20-3 whose view is adjacent. is doing. The CPUs of the monitoring side terminals 20-1, 20-2, 20-3 start, take in and decode the video information sent by the assigned cameras, perform image processing on these (S41), and detect the presence of a moving object (S42).

  If a moving object cannot be detected in step S42, it is detected whether or not the “monitoring object is present” is notified from the adjacent monitoring side terminal (S43), and the notification that “the moving object is present” has arrived. If so, the process returns to step S41 to continue the processing. Further, when a moving body is detected in step S42 or when a notification of “with moving body” is detected in step S43, image processing is performed according to the processing content determined in the case of detecting a moving body (S44). ) To detect whether or not the camera has moved to the adjacent camera field of view, based on whether it has moved in the direction of the adjacent camera beyond predetermined position information (for example, information such as the number of pixels in the horizontal direction from the edge of the screen). (S45).

  When it is detected in step S45 that the moving body has not moved to the adjacent camera field of view, the process returns to step S44 and the process is continued. In step S45, it is detected that the moving body has moved to the adjacent camera field of view. In step S46, the monitoring terminal corresponding to the adjacent camera is notified that there is a moving object (S46), and the process returns to step S41. As a result, the monitoring terminals 20-1, 20-2, and 20-3 can perform image processing according to the processing content determined in the case of moving object detection from an early stage. Decoding using not only I frames but also P frames and B frames, decoding from black and white images to color images, changing processing algorithms and combining several algorithms for image processing It is possible to improve the processing accuracy by, for example.

  FIG. 11 shows a fourth embodiment of the monitoring system according to the present invention. In this monitoring system, the IP cameras 11A-1, 11A-2, and 11A-3, which are a plurality of video transmission units, each include a zoom mechanism and a turning mechanism, and camera control means provided in the monitoring terminal 20A. The zoom mechanism and the turning mechanism in the IP cameras 11A-1, 11A-2, and 11A-3 are controlled by the control information from 24 to perform zooming and turning.

  A patrol data table 25 in which information for allowing the camera control means 24 to zoom and turn the IP cameras 11A-1, 11A-2, 11A-3 is provided in the monitoring terminal 20A. As shown in FIG. 12, the patrol data table 25 stores data in which the turning angle, turning speed, and zoom ratio data in each phase are associated with each other in correspondence with the camera ID. The camera with camera ID a in FIG. 12 corresponds to IP camera 11A-1, and the camera with camera ID b corresponds to IP camera 11A-2. In the camera with camera ID a, in phase F1, the turn from 0 degree to 90 degrees is performed at a speed (1 degree per second), the zoom ratio is kk, and in phase F2, the turn from 91 degrees to 95 degrees At a speed (0.3 degrees per second), indicating that the zoom ratio is tt. Further, in the camera with camera ID b, in phase F1, a turn from 0 degree to 190 degrees is performed at a speed (0.3 degrees per second), the zoom ratio is kk, and in phase F2, 191 degrees to 295. It turns that the turning to the degree is performed at the speed (1 degree per second), and the zoom ratio is tt.

  The CPU of the monitoring terminal 20A decodes the video information sent by the IP cameras 11-1, 11-2, and 11-3, and performs processing corresponding to the flowchart shown in FIG. The operation will be described with reference to the flowchart shown in FIG. The CPU of the monitoring terminal 20A starts and sends control information from the phase F1 to the IP cameras 11-1, 11-2, 11-3 according to the information in the patrol data table 25, and controls the turning and zooming. Information is captured and decoded to perform image processing (S51), and it is detected whether an event has occurred (S52). In this image processing, capturing and decoding of video information from the IP cameras 11-1, 11-2, and 11-3 and image processing are performed by equally allocating resources.

  If no event occurrence is detected in step S52, the process returns to step S51 for processing. On the other hand, if an event occurrence is detected in step S52, a predetermined value (for example, the maximum value) for the corresponding camera is detected. ) Until the zoom speed is reduced to a predetermined value (for example, minimum value or stop), camera control is performed for a predetermined second, and video information is captured and decoded. Image processing is performed (S53).

  Following step S53, the turning angle, speed, and zoom ratio in step S53 are stored, and compared with the information in the patrol data table 25 (S54), it is checked whether the turning angle, speed, and zoom ratio match. (S55). If it is detected in step S55 that the values match, the process returns to step S51 to continue the process. If a mismatch is detected in step S55, the patrol data table 25 reflects the turning angle, speed, and zoom ratio. (S56), the process returns to step S51 and continues. In this way, patrol monitoring by the camera is performed with the turning speed and zoom ratio appropriate for the turning place (turning range) where the event occurs. In the present embodiment, the turning angle, speed, and zoom ratio are reflected in the patrol data table 25 by one event occurrence. However, the condition is that a predetermined number of events have occurred at the same turning angle. You may make it reflect as.

  FIG. 14 shows a fifth embodiment of the monitoring system according to the present invention. In this monitoring system, three monitoring terminals 20A-1, 20A-2, and 20A-3 are connected to the network 10. The monitoring terminals 20A-1, 20A-2, 20-3 are provided with camera control means 24-1, 24-2, 24-3 in addition to the configuration of FIG. The IP camera 11A-1, 11A-2, 11A-3 can take an image of the same moving body by turning. For example, when the moving body captured by the IP camera 11A-1 is captured, the IP camera 11-2 Each camera is arranged in a situation where the same moving body can be imaged by performing panning of the IP camera 11-3. Further, the CPUs of the monitoring side terminals 20A-1, 20A-2, and 20A-3 decode the video information sent by the IP cameras 11A-1, 11A-2, and 11A-3 assigned to them, respectively. Since the processing corresponding to the flowchart shown in FIG. 15 is performed, the operation of this system will be described below with reference to the flowchart shown in FIG.

  Here, a case where a moving body is captured by the IP camera 11A-1 and the IP camera 11-2 and the IP camera 11-3 are panned will be described, and the monitoring side terminals 20A-1, 20A-2, and 20A-3 are described. Each hold the address of another monitoring terminal. The CPUs of the monitoring side terminals 20A-1, 20A-2, and 20A-3 are started, take in and decode the video information sent by the assigned cameras, perform image processing on these (S61), and detect the presence of a moving object. (S62).

  If a moving object cannot be detected in step S62, it is detected whether or not a “moving object exists” notification has been made from another monitoring terminal (S63), and a “moving object exists” notification has arrived. If so, the process returns to step S61 to continue the processing. If a moving body is detected in step S62, the presence of the moving body is notified to the monitoring side terminals (in this case, 20A-2 and 20A-3) corresponding to a predetermined camera (S67), and the IP camera 11A is detected. The video information sent from the IP camera 11A-1 is fetched and decoded while performing turning control on -1 according to the moving body, and image processing is performed (S68). At this time, the turning of the camera is controlled so that the moving object is reflected in the center area of the screen.

  As a result of the turning control in step S68, it is determined by determining whether or not the tracking range has been exceeded based on the turning angle (S65). If the tracking range is not exceeded, the processing in step S68 is continued, and if the tracking range is exceeded, IP camera 11A-1 On the other hand, turning control for returning to the home position is performed (S66), and the process returns to step S61.

  In step S62, when the moving body cannot be detected, it is detected whether the notification of “moving body” is given from another monitoring terminal (S63), and the notification of “moving body is present” has arrived. If so, the process returns to step S61 to continue the processing. If a moving object is detected in step S62, the IP camera 11A-1 is controlled to turn by an angle determined when the moving object is detected, and the turning control is performed according to the moving object and image processing is performed. Perform (S64). Subsequent to step S64, it is determined by determining whether the tracking range has been exceeded based on the turning angle (S65). If the tracking range has not been exceeded, the processing in step S68 is continued. The turning control to return to the home position is performed (S66), and the process returns to step S61.

  The CPUs of the monitoring side terminals 20A-2 and 20A-3 branch to YES in step S63 and continue the processing in response to the notification “There is a moving object” from the monitoring side terminal 20A-1, and therefore the monitoring side terminal 20A-1 When the “moving object is detected” is detected, the IP cameras 11A-2 and 11A-3 are panned and tracking of the moving object is started. Therefore, when performing image processing, only I frames are decoded. In addition, decoding using P frames and B frames, decoding from black and white images to color images, changing processing algorithms and combining several algorithms for image processing, etc. The processing accuracy can be improved.

  In the sixth embodiment, as shown in FIG. 16, for example, the monitoring terminal 20-1 in the monitoring system shown in FIG. 6 includes a processed video creation means 26, an event occurrence notification image provision means 27, and a video distribution means 28. It is configured. The other means provided in the monitoring terminal 20-1 has been described in the embodiment in FIG. 6, and is not shown here. In this monitoring system, a client terminal 40 (here, there is one, but a plurality of client terminals) to which the processed video information is distributed is connected to the network 10, and the IP cameras 11-1, 11- Connected to the network 10 is a video storage server 50 which is a video storage means for acquiring and storing video information sent from the video sending unit composed of 2 and 11-3 from the network 10.

  The processed video creation means 26 creates a panoramic video based on video information from several cameras, and creates a bird's-eye video showing the moving state of the moving body based on video information from several cameras. It is. Further, the event occurrence notification image providing means 27 sends an event occurrence notification image in which information for acquiring the video before and after the occurrence of the event from the video storage means is set to the image information at the time of occurrence of the event to the client. The video distribution means 28 includes a panoramic video created by the processed video creation means 26, image information of a bird's eye view video, event occurrence notification image information by the event occurrence notification image providing means 27, etc. This is distributed to the terminal 40.

  The processed video creation means 26 is configured to create a panoramic video when the angle of view (background) of the IP cameras 11-1, 11-2, 11-3 is similar, for example, and the IP cameras 11-1, 11- 2 and 11-3 are configured to create a bird's-eye view video when the angle of view (background) is different. Since the processed video creation means 26 is realized by the CPU of the monitoring terminal 20-1 executing a program corresponding to the flowcharts shown in FIGS. 17 and 18, the operation based on the flowcharts shown in FIGS. Will be explained.

  When there is a moving object detection, a panoramic video is created by a program corresponding to the flowchart shown in FIG. First, the moving speed of the moving object is obtained by the image processing of the image processing means 22, the acquisition time of the video information sent by each camera is determined (S71), and each IP camera 11-1, 11-2, 11 is acquired at the acquisition time. The image information sent from -3 is taken in, the image information of the still image is obtained, and the image is synthesized by overlapping the predetermined length portions on the side portions of each screen, for example, so that the background is connected (S72).

  In acquisition of video information sent from each IP camera 11-1 (ID: a), 11-2 (ID: b), 11-3 (ID: c) in step S72, as shown on the left side of FIG. When an a image, a camera b image, and a camera c image are obtained, they are combined in the horizontal direction, and a panoramic video P1 is obtained by superimposing dashed arrow images indicating the traveling direction. The panoramic video thus obtained is distributed to the client terminal 40 by using the address of the client terminal 40 held in advance by the video distribution means 28 (S73).

  When creating a bird's-eye view video, if there is a moving object detection, a program corresponding to the flowchart shown in FIG. 18 is executed. First, the moving speed of the moving body is obtained by image processing of the image processing means 22, the acquisition time of video information sent by each camera is determined, and from each IP camera 11-1, 11-2, 11-3 at the acquisition time. The video information to be sent is captured and decoded (S81). Next, in the decoded image corresponding to each camera, processing for detecting the distance and position from each camera to the moving body from the coordinates on the screen is performed, and each distance and position is converted into position coordinates on the bird's-eye view screen (S82). . For example, a conversion table prepared in advance is used for the distance detection and the conversion from the distance and position to the position coordinates of the bird's-eye view.

  At the position coordinates determined in step S82, the moving object mark M is arranged as shown in the bird's-eye view video P2 in FIG. 19, and a broken-line arrow image indicating the moving direction of the moving body is arranged, and the bird's-eye view video P2 is arranged. Is completed, and this bird's-eye view video P2 is distributed by the video distribution means 28 to the client terminal 40 using the address of the client terminal 40 held in advance (S83).

  The monitoring terminal 20-1 transfers Web data for displaying, for example, video information of a camera (not shown) to the client terminal 40 as a WWW server, and the event occurrence display E of the monitoring screen H shown in FIG. It has a function of displaying what is not displayed on the display of the client terminal 40. When an event occurs, the CPU of the monitoring terminal 20-1 activates a program corresponding to the flowchart shown in FIG. 20, and the event occurrence notification image providing means 27 provides the image information of the event occurrence notification image.

  That is, when an event occurs and a program corresponding to the flowchart shown in FIG. 20 is started, the CPU stores the decoded image information (displayed on the event occurrence display E) at the time of the event in the video storage server 50. The program (link destination address, etc.) for retrieving the video in which the video information of about N seconds related to the occurrence of the event is stored is chained and sent to the client terminal 40 (S91). Thereby, the Web browser of the client terminal 40 displays the monitoring screen H of FIG. The event occurrence display E portion is chained with a program for extracting a video in which video information about N seconds related to the event occurrence is accumulated, and when the event occurrence display E portion is clicked with a mouse, the event occurrence indication E Video information about N seconds is sent from the video storage server 50, and a video image of about N seconds related to the occurrence of the event is displayed on the display of the client terminal 40. As described above, according to this embodiment, the panoramic video, the bird's-eye video, and the video of N seconds related to the occurrence of the event can be reproduced in the client terminal as shown in FIG. Is possible. The configuration of the sixth embodiment can be adopted together with the configurations of other embodiments such as the first embodiment, and more efficient monitoring can be performed.

The figure which shows the structure of the monitoring system which concerns on 1st Example in this invention. The figure which shows the other structural example of the image | video transmission part used for the monitoring system of this invention. The flowchart for demonstrating operation | movement of the monitoring system which concerns on 1st Example in this invention. The figure which shows the task structure for the parallel processing by the monitoring system which concerns on 1st Example in this invention. The figure for demonstrating the transition of the resource allocation by the monitoring system which concerns on 1st Example in this invention. The figure which shows the structure of the monitoring system which concerns on 2nd Example in this invention. The figure which shows the content of the process request information used with the monitoring system which concerns on 2nd Example in this invention. The flowchart for demonstrating operation | movement of the monitoring system which concerns on 2nd Example in this invention. The flowchart for demonstrating operation | movement of the monitoring system which concerns on 2nd Example in this invention. The flowchart for demonstrating operation | movement of the monitoring system which concerns on 3rd Example in this invention. The figure which shows the structure of the monitoring system which concerns on the 4th Example in this invention. The figure which shows the content of the data table for patrols used with the monitoring system which concerns on the 4th Example in this invention. The flowchart for demonstrating operation | movement of the monitoring system which concerns on the 4th Example in this invention. The figure which shows the structure of the monitoring system which concerns on the 5th Example in this invention. The flowchart for demonstrating operation | movement of the monitoring system which concerns on the 5th Example in this invention. The figure which shows the structure of the monitoring system which concerns on the 6th Example in this invention. The flowchart for demonstrating operation | movement of the monitoring system which concerns on the 6th Example in this invention. The flowchart for demonstrating operation | movement of the monitoring system which concerns on the 6th Example in this invention. The figure for demonstrating the process image | video production | generation operation | movement in the monitoring system which concerns on the 6th Example in this invention. The flowchart for demonstrating operation | movement of the monitoring system which concerns on the 6th Example in this invention. The figure for demonstrating event generation alerting image creation operation | movement in the monitoring system which concerns on the 6th Example in this invention. The figure which shows the structure of the monitoring system which concerns on a prior art example.

Explanation of symbols

10 network
11-1 to 11-3, 11A-1 to 11A-3 IP camera
12 Analog camera
13 Encoder
20, 20-1 to 20-3, 20A, 20A-1 to 20A-3 Monitoring side terminal
21, 211-1 to 21-3 decoder
22 Image processing means
23 Display control means
24 Camera control means
25 Data table for patrol
26 Processed image creation means
27 Event occurrence notification image providing means
28 Video distribution means
30, 30-1 to 30-3 Image display unit
40 client terminals
50 video storage server

Claims (13)

A plurality of video transmission units including a camera that captures an object to be imaged, and outputting encoded video information that can be network-transmitted for the captured video;
A network to which the plurality of video transmission units are connected;
A decoder connected to the network for capturing at least an I frame included in the video information transmitted by the plurality of video transmission units, and capturing a B frame and a P frame as necessary, and converting them into digital image information for one screen; ,
Image analysis is performed in parallel processing in which a plurality of image processings for detecting an object is performed in parallel on at least one I-frame image which is digital image information corresponding to a plurality of video transmission units converted by the decoder, and event occurrence and disappearance The image processing means for detecting the digital image information corresponding to one video transmission unit related to parallel processing in the event of an event transfer to another image processing means, and the I, B, P frames are An image processing means for performing the processing used and returning the processing to the original parallel processing when the event disappears . It said image processing means has a plurality of tasks, claims, characterized in that to change the allocation ratio of CPU resources to each task related to digital image information according to the transmission of the corresponding picture sending unit in accordance with the event occurrence annihilation The monitoring system according to 1.   The monitoring system according to claim 1, wherein the image processing unit is realized by a single device.   The monitoring system according to claim 1, wherein the image processing unit is provided in each of a plurality of apparatuses.   The monitoring system according to claim 1, wherein the image processing unit notifies the image processing unit that processes digital image information of a corresponding camera according to an analysis result. The camera includes a zoom mechanism and a turning mechanism,
6. The camera control unit according to claim 1, further comprising: a camera control unit that controls a zoom mechanism and a turning mechanism of a corresponding camera according to an analysis result by the image processing unit. Monitoring system. A patrol data table in which turning speed information and zoom information are associated with turning angle information;
The monitoring system according to claim 6, wherein the camera control unit controls a zoom mechanism and a turning mechanism in a corresponding camera based on information in the patrol data table.   6. The monitoring system according to claim 5, wherein the camera control means receives the event occurrence information sent via the network, and controls the zoom mechanism and the turning mechanism in the corresponding camera according to the event occurrence information. . While the image processing means performs notification according to the analysis result to the camera control means provided in each of the plurality of devices,
7. The monitoring system according to claim 6, wherein the camera control means in the apparatus that has received the notification controls the zoom mechanism and the turning mechanism of the corresponding camera in response to the notification.   The monitoring system according to any one of claims 1 to 9, further comprising processed image creation means for creating a panoramic image based on video information obtained by several cameras.   10. The modified video creation means for creating a bird's-eye view video showing a moving state of a moving body based on video information obtained by several cameras. The monitoring system described in. Video storage means for fetching and storing video information sent from the video sending unit from the network;
An event occurrence notification image providing means for sending an event occurrence notification image in which information for acquiring images before and after the occurrence of an event from the image storage means is set to image information at the time of occurrence of the event, to the client. The monitoring system according to any one of claims 1 to 9. A decoder for converting optionally write Mutotomoni takes at least I-frames included in the video information to the B-frame P-frame also takes Write-one screen digital image information,
Image processing means for performing image analysis in parallel processing for executing a plurality of parallel image processings for detecting an object for at least one I-frame image that is digital image information converted by the decoder, and detecting occurrence and disappearance of events. In the event of an event, the processing of digital image information corresponding to one video transmission unit related to parallel processing is transferred to another image processing means, processing using I, B, and P frames is performed, and the event disappears And an image processing means for returning the processing to the original parallel processing .