JP6311809B2 - Image processing system, image processing apparatus, image processing method, and image processing program - Google Patents

Description

  The present disclosure relates to an image processing system, an image processing apparatus, an image processing method, and an image processing program for tracking a registered tracking target on a time-series image.

  In recent years, image processing techniques for discriminating human actions from videos have been developed. This image processing technology is applied to, for example, an image processing apparatus for watching the behavior of a care recipient who needs care. Regarding such an image processing technique, Japanese Patent Application Laid-Open No. 2002-230533 (Patent Document 1) states that “image processing that can accurately determine an entry event and an exit event using a boundary side that is a lower side of the bed outer shape”. Apparatus ". Japanese Patent Laying-Open No. 2012-30042 (Patent Document 2) states that “by automatically detecting a region to be monitored with reference to a bed, it is possible to accurately detect an action of a person to be monitored with high reproducibility. Is disclosed.

JP 2002-230533 A JP 2012-30042 A

  The image processing apparatus disclosed in Patent Document 1 determines that the person is in the floor when detecting the movement of the person from the floor area to the bed area, and moves the person from the bed area to the floor area. If detected, it is determined that the person is getting out of bed. The determination process for entering and leaving the bed is based on the premise that the bed area is accurately set in advance. For this reason, when the bed is moved, the image processing apparatus cannot accurately determine the behavior of the person. Therefore, there is a demand for an image processing apparatus that prompts the administrator to make a setting again when the setting used in the action determination process becomes incorrect.

  The monitoring device disclosed in Patent Literature 2 extracts the position of the bed using the distance image output by the distance image sensor, and detects a region occupied by a person within and outside the range of the bed. The monitoring device determines a person's behavior with respect to the bed based on a combination of the detected bed and the detected person's area. The generation of the distance image by the distance image sensor is realized by using an active method or a plurality of cameras. Therefore, when the distance image sensor is used, the cost increases. In addition, processing time is a problem for generating a distance image, and it is difficult to always execute the action determination process.

  The present disclosure has been made to solve the above-described problems, and its purpose is to relatively prompt the administrator to make a setting again when the setting used in the behavior determination process becomes incorrect. An object of the present invention is to provide an image processing system that can be realized with a simple configuration. An object in another aspect is to provide an image processing apparatus that can realize, with a relatively simple configuration, prompting an administrator to make a setting again when a setting used in the behavior determination process becomes incorrect. Still another object of the present invention is to provide an image processing method capable of realizing, with a relatively simple configuration, prompting an administrator again for a setting when a setting used in the behavior determination process becomes incorrect. Still another object of the present invention is to provide an image processing program capable of realizing, with a relatively simple configuration, prompting the administrator to set again when the setting used in the behavior determination process becomes incorrect.

According to one embodiment, the image processing system captures a subject from the same viewpoint as the reference image, a receiving unit for receiving a setting of an area including the object with respect to a reference image obtained by capturing the subject. Based on the positional relationship between the person detection unit for detecting the person area from the obtained input image, the area including the object in the input image, and the person area in the input image, the action of the person in the person area Comparing the image information of the area including the object in the reference image with the image information of the area including the object in the input image, or the vicinity of the area including the object in the reference image A shift detection unit for comparing the image information with image information in the vicinity of the region including the object in the input image and detecting that the object is shifted from the region including the object in the input image; Object displacement is detected When it is, without determining the behavior of a person, and a notification unit for notifying that the deviation object in the input image from the region containing the set object.

  Preferably, the reception unit receives the setting of the region including the object by receiving at least one of a plurality of point settings for the reference image, a plurality of line settings for the reference image, and a surface setting for the reference image.

  Preferably, the reception unit further receives setting of the orientation of the object in the reference image.

  Preferably, the action determination unit determines a person's action based on at least one of a size of a person area occupying an area including an object of the input image and a size of a person area occupying outside the area including an object of the input image. Determine.

  Preferably, the object is a bedding. The person detection unit sequentially detects a person from a plurality of time-series input images. The action determination unit sequentially calculates the size of the person area occupying outside the area including the object of the input image, and the person is woken up based on the fact that the size of the person area is sequentially larger than a predetermined threshold. It is determined that

  Preferably, the object is a bedding. The person detection unit sequentially detects a person from a plurality of time-series input images. The action determination unit sequentially calculates the size of the person area occupying the area including the object of the input image, and the person gets out of bed based on the fact that the sequentially calculated person area size is smaller than a predetermined threshold. It is determined that

According to another aspect, the image processing apparatus obtains a reception unit for receiving a setting of a region including an object with respect to a reference image obtained by photographing the subject, and photographs the subject from the same viewpoint as the reference image. A person detection unit for detecting a person area from the input image, an area including an object in the input image, and a person's action in the person area based on a positional relationship between the person area in the input image. Compare the image information of the area including the object in the reference image with the image information of the area including the object in the input image, or an image near the area including the object in the reference image A displacement detection unit for comparing information with image information in the vicinity of the region including the object in the input image and detecting that the object is shifted from the region including the object in the input image; and an object in the input image When a deviation is detected To, without determining the behavior of a person, and a notification unit for notifying that the deviation object in the input image from the region containing the set object.

According to yet another aspect, the image processing method includes a step of receiving a setting of a region including an object with respect to a reference image obtained by photographing the subject, and an input obtained by photographing the subject from the same viewpoint as the reference image. Detecting a person area from the image; determining a person's action in the person area based on a positional relationship between the area including the object in the input image and the person area in the input image; Compare the image information of the area including the object in the image with the image information of the area including the object in the input image, or the area including the object in the input image and the image information near the area including the object in the reference image by comparing the image information in the neighborhood, discrimination detecting that are out of the object from the area including an object in the input image, when the displacement of the object is detected in the input image, the behavior of the person Not in, and a step of notifying that the deviation object in the input image from the region containing the set object.

According to yet another aspect, the image processing program captures a subject from the same viewpoint as the reference image, and a step of accepting a setting of an area including the object with respect to a reference image obtained by photographing the subject. Detecting a person area from the obtained input image; determining a behavior in the person area based on a positional relationship between the area including the object in the input image and the person area in the input image; Compare the image information of the area including the object in the reference image with the image information of the area including the object in the input image, or include the image information in the vicinity of the area including the object in the reference image and the object in the input image. by comparing the image information of a region near a step of detecting that the shift object from the area including an object in the input image, when the displacement of the object is detected in the input image, Without determining the behavior of the object, and a step of notifying that the deviation object in the input image from the region containing the set object.

  According to yet another aspect, the image processing program captures a subject from the same viewpoint as the reference image, and a step of accepting a setting of an area including the object with respect to a reference image obtained by photographing the subject. A step of detecting a person from the obtained input image, a step of determining a person's action with respect to the object based on a positional relationship between the area in the input image and the person area in the input image; Comparing the image information of the region or the vicinity of the region with the image information of the region or the region in the input image to detect that the object is displaced from the region, and detecting the displacement of the object And a step of notifying that the object is displaced from the set area.

  In one aspect, the image processing apparatus can realize, with a relatively simple configuration, prompting the administrator to make a setting again when the setting used in the behavior determination process becomes incorrect.

  The above and other objects, features, aspects and advantages of the present invention will become apparent from the following detailed description of the present invention taken in conjunction with the accompanying drawings.

It is a figure which shows an example of a structure of the image processing system according to 1st Embodiment. It is a figure which shows the setting image for setting the area | region of a bed, and the input image which is the object which detects the shift | offset | difference of a bed. It is a figure which shows the screen transition diagram of the image processing system according to 1st Embodiment. It is a figure which shows an example of a main screen. It is a figure which shows an example of a setting mode top screen. It is a figure which shows an example of an area | region setting screen. It is a figure which shows an example of a normal time screen. It is a figure which shows an example of the screen at the time of alerting | reporting. It is a figure which shows an example of the screen at the time of abnormality. It is a block diagram which shows an example of a function structure of the image processing system according to 1st Embodiment. It is a flowchart which shows the preliminary process performed before starting an action discrimination | determination process. It is the conceptual diagram which showed schematically the preliminary process performed before starting an action discrimination | determination process. It is a flowchart which shows this process performed with the image processing system according to 1st Embodiment. It is a flowchart which shows the abnormality detection process performed with the image processing system according to 1st Embodiment. It is the conceptual diagram which showed the abnormality detection process schematically. It is a flowchart which shows the action discrimination | determination process performed with the image processing system according to 1st Embodiment. It is the conceptual diagram which showed roughly the process which extracts a person area | region from an input image. It is a flowchart which shows the wake-up discrimination | determination process performed with the image processing system according to 1st Embodiment. It is a flowchart which shows the bed leaving determination process performed with the image processing system according to the first embodiment. It is a block diagram which shows the main hardware constitutions of the image processing system according to 1st Embodiment. It is a flowchart which shows the abnormality detection process performed with the image processing system according to 2nd Embodiment.

  Embodiments according to the present invention will be described below with reference to the drawings. In the following description, the same parts and components are denoted by the same reference numerals. Their names and functions are also the same. Therefore, detailed description thereof will not be repeated. Each embodiment and each modified example described below may be selectively combined as appropriate.

<First Embodiment>
[Configuration of Image Processing System 300]
The configuration of the image processing system 300 will be described with reference to FIG. FIG. 1 is a diagram illustrating an example of the configuration of an image processing system 300.

  The image processing system 300 is used for watching the behavior of the care recipient 10, for example. As shown in FIG. 1, the image processing system 300 includes an indoor terminal 100 as an image processing apparatus and a management server 200. The indoor terminal 100 and the management server 200 are connected to each other via the network 400.

  The indoor terminal 100 is installed in, for example, a medical facility, a care facility, or a home. The indoor terminal 100 includes a camera 105 (FIG. 20) described later. FIG. 1 shows a state where the camera 105 is photographing the care receiver 10 and the bed 20 from the ceiling. The indoor terminal 100 determines the behavior of the care receiver 10 based on a time-series image (video) obtained from the camera 105. As an example, actions that can be discriminated by the indoor terminal 100 include wake-up of the cared person 10 and leaving the cared person 10. Here, “wake-up” refers to an action from when the care recipient 10 wakes up on the bed 20 until he / she stands up. Further, “getting out of bed” refers to an action in which the care recipient 10 leaves the bed 20.

  When the indoor terminal 100 detects that the cared person 10 wakes up, the indoor terminal 100 transmits information indicating that the cared person 10 has woken up to the management server 200. When the management server 200 receives the information, the management server 200 notifies the caregiver that the care receiver 10 has woken up. Further, when the indoor terminal 100 detects that the care receiver 10 has left the bed, the indoor terminal 100 transmits information indicating that the care receiver 10 has left the floor to the management server 200. When the management server 200 receives the information, the management server 200 notifies the caregiver that the care receiver 10 has left the bed. Thereby, the caregiver can assist the care receiver 10 to get up from the bed 20, or can prevent risks such as a fall that occurs when the care receiver 10 gets up or leaves.

  1 illustrates an example in which the image processing system 300 includes one indoor terminal 100, the image processing system 300 may include a plurality of indoor terminals 100. 1 illustrates an example in which the image processing system 300 includes one management server 200, the image processing system 300 may include a plurality of management servers 200.

[Processing Overview of Image Processing System 300]
With reference to FIG. 2, an outline of processing of image processing system 300 according to the first embodiment will be described. FIG. 2 is a diagram illustrating a setting image 30A for setting an area including the bed 20 and an input image 30B that is a target for detecting a shift of the bed 20. As illustrated in FIG.

  The image processing system 300 accepts the setting of the bed area 40 including the bed 20 with respect to the setting image 30A (reference image) obtained by photographing the subject. The “subject” here refers to an object included in the field of view of the camera 105 described later. That is, the subject includes an object such as a bed, a care recipient to be monitored, and the like. For example, the image processing system 300 receives the setting of the bed area 40 by receiving the settings of the points 41A to 41D for the setting image 30A. As an example, the administrator of the image processing system 300 inputs points 41 </ b> A to 41 </ b> D corresponding to the four ends of the bed 20 to the setting image 30 </ b> A by a mouse operation or a touch operation. When receiving the setting of the bed area 40, the image processing system 300 stores information for specifying the bed area 40 in the setting image 30A (for example, coordinate values of the four corners of the bed area 40).

  When the input image 30B is acquired from a camera 105 (FIG. 20) described later, the image processing system 300 detects the person 12 from the input image 30B. The person 12 corresponds to the care receiver 10 (FIG. 1) described above. The input image 30B is an image obtained by photographing a subject from the same viewpoint as the setting image 30A. Typically, the shooting time of the input image 30B is later than the shooting time of the setting image 30A.

  The image processing system 300 determines the behavior of the person 12 with respect to the bed 20 based on the positional relationship between the bed area 40 in the input image 30B and the person area in the input image 30B. The bed area 40 of the setting image 30A and the bed area 40 of the input image 30B are located at the same place in both images. The actions that can be discriminated by the image processing system 300 include, for example, getting up and getting out of the person 12. Details of the action determination process will be described later (see FIG. 16).

  The image processing system 300 compares the image information of the bed area 40 in the setting image 30A and the image information of the bed area 40 in the input image 30B, or inputs image information in the vicinity of the bed area 40 in the setting image 30A. The image information in the vicinity of the bed area 40 in the image 30B is compared to detect that the bed 20 is displaced from the bed area 40. When detecting the displacement of the bed 20, the image processing system 300 notifies that the bed 20 is displaced. For example, the displacement of the bed 20 is notified by displaying the message 50.

  Thus, when the bed 20 is displaced from the initially set bed area 40, the image processing system 300 automatically detects the displacement of the bed 20, and notifies the administrator that the bed 20 has been displaced. Since the bed area 40 set by the administrator is used for the action determination process of the person 12, the displacement of the bed 20 greatly affects the accuracy of the action determination process. The image processing system 300 can prompt the administrator to correct the bed area 40 by notifying the administrator that the bed 20 is displaced. As a result, the image processing system 300 can prevent the accuracy of the action determination process from being lowered, and can reduce the burden on the caregiver by being notified of getting up or getting out of bed by mistake.

  In addition, although the example which the image processing system 300 alert | reports the shift | offset | difference of a bed was demonstrated above, the notification object of a shift | offset | difference is not limited to the bed which is bedding. For example, the notification target of deviation includes bedding other than a bed such as a futon, a chair, a curtain, a television, and other objects used by the person 12. In order to detect that these objects are displaced, the administrator may set an area including an object for which notification of displacement is desired for the setting image 30A.

[Screen Transition of Image Processing System 300]
Examples of screens displayed on the image processing system 300 will be described with reference to FIGS. FIG. 3 is a diagram showing a screen transition diagram of the image processing system 300.

  The image processing system 300 displays the main screen 310 as an initial screen when the image processing program according to the present embodiment is executed. The administrator can transition the main screen 310 to the setting mode top screen 320 or the normal screen 340. The administrator can transition the setting mode top screen 320 to the main screen 310 or the area setting screen 330. The administrator can transition the area setting screen 330 to the setting mode top screen 320. The administrator can transition the normal screen 340 to the main screen 310, the alerting screen 350, or the abnormal screen 360. The administrator can change the notification time screen 350 to the normal time screen 340 or the abnormal time screen 360. The administrator can transition the abnormal time screen 360 to the normal time screen 340 or the notification time screen 350.

  Hereinafter, screen examples of the main screen 310, the setting mode top screen 320, the region setting screen 330, the normal time screen 340, the alerting time screen 350, and the abnormal time screen 360 will be described in order.

(A) Main screen 310
FIG. 4 is a diagram illustrating an example of the main screen 310. The image processing system 300 displays the main screen 310 as an initial screen when the image processing program according to the present embodiment is executed.

  The main screen 310 includes a button 312 for accepting the start of the action determination process and a button 314 for opening a setting screen related to the action determination process. When detecting that the button 312 is pressed, the image processing system 300 displays a normal screen 340. In addition, when the image processing system 300 detects that the button 314 is pressed, the image processing system 300 displays a setting mode top screen 320.

(B) Setting mode top screen 320
FIG. 5 is a diagram illustrating an example of the setting mode top screen 320. The setting mode top screen 320 accepts parameter settings related to behavior determination processing. For example, the setting mode top screen 320 receives a frame rate setting of the camera 105 described later. In addition, the setting mode top screen 320 receives the setting of the brightness of the image output from the camera 105. Furthermore, the setting mode top screen 320 accepts the setting of a parameter indicating sensitivity for notifying the detected behavior of the cared person. Furthermore, the setting mode top screen 320 receives the setting of the height of the ceiling where the camera 105 is installed. When the “update button” on the setting mode screen is pressed, each parameter is reflected in the image processing system 300.

  The setting mode top screen 320 includes a button 322 for opening the area setting screen 330 and a button 324 for opening the main screen 310. When the image processing system 300 detects that the button 322 is pressed, the image processing system 300 displays an area setting screen 330. The image processing system 300 displays the main screen 310 when it detects that the button 324 is pressed.

  The setting mode top screen 320 may accept setting of other parameters. For example, the setting mode top screen 320 accepts settings related to the camera 105 such as a parameter related to the contrast of the input image, a parameter related to the zoom adjustment of the camera, and a parameter related to the pan / tilt adjustment of the camera. In addition, the setting mode top screen 320 may accept settings such as a compression rate of an image transmitted from the indoor terminal 100 to the image processing system 300. In addition, the setting mode top screen 320 may accept a setting such as a time zone for determining an action such as getting up or getting out of bed.

(C) Area setting screen 330
FIG. 6 is a diagram illustrating an example of the area setting screen 330. The area setting screen 330 receives the setting of the bed area 40 for the setting image 30A (FIG. 2) described above. The set bed area 40 is used in the action determination process.

  The area setting screen 330 receives the setting of the bed area 40 by receiving the settings of the points 41A to 41D, for example. As an example, the points 41A to 41D are input by a pointer 332 that is interlocked with a mouse operation. The image processing system 300 stores information (for example, coordinates) for specifying the bed area 40 in the setting image 30A based on the reception of the operation for saving the bed area 40 set by the administrator. .

  In FIG. 6, the example of setting the points 41 </ b> A to 41 </ b> D has been described as the method for setting the bed area 40, but the bed area 40 may be set by other methods. For example, the image processing system 300 receives the setting of the bed area 40 by receiving a plurality of line settings for the area setting screen 330. In this case, the administrator sets four lines on the area setting screen 330 so as to surround the bed 20. As another method, the image processing system 300 accepts the setting of the bed area 40 by accepting the setting of the surface for the area setting screen 330. In this case, the administrator designates a range in which the bed 20 is shown by a drag operation on the area setting screen 330. Thus, as a setting method of the bed area 40, any method capable of designating the bed area 40 can be adopted.

  In FIG. 6, an example of setting a rectangular area as a method of setting the bed area 40 has been described. However, the bed area 40 may be set in another shape. For example, the bed area 40 may be set in other shapes such as a circle, an ellipse, and a polygon (for example, a hexagon).

  Furthermore, in FIG. 6, an example in which the bed area 40 is set with the pointer 332 has been described, but the bed area 40 may be set by other operations such as a touch operation.

(D) Normal screen 340
FIG. 7 is a diagram illustrating an example of the normal screen 340. The normal screen 340 is a screen that is displayed when the person to be monitored is taking a non-hazardous action (for example, going to bed) while the action determination process is performed by the image processing system 300. As an example, the image processing system 300 displays a video obtained by photographing a person to be monitored as a normal screen 340 as it is.

(E) Notification screen 350
FIG. 8 is a diagram illustrating an example of a notification time screen 350. The notification time screen 350 is a screen that is displayed when the person to be monitored takes an action that he / she wants to report, such as a dangerous action, during execution of the action determination process by the image processing system 300.

  As shown in FIG. 8, the image processing system 300 notifies the caregiver of getting out of the care receiver 10 based on detecting that the care receiver 10 has left the bed. In one aspect, the image processing system 300 notifies the caregiver of getting out of the care receiver 10 by a message 352. In another aspect, the image processing system 300 notifies the caregiver of getting out of the care receiver 10 by sound such as voice. Thereby, the image processing system 300 can prevent risks such as a fall caused by the care recipient 10 getting out of bed.

  In addition, the action of the care receiver 10 to be notified is not limited to getting out of bed. For example, the behavior to be reported includes wake-up of the care receiver 10, entry of the care receiver 10, and other actions that cause danger to the care receiver 10.

(F) Abnormal screen 360
FIG. 9 is a diagram illustrating an example of the abnormal time screen 360. The abnormal time screen 360 is displayed when it is detected that the setting of the object region related to the action determination process is not correct during execution of the below-described main process (see FIGS. 13 to 19) by the image processing system 300. It is a screen. For example, when the image processing system 300 detects that the bed 20 has shifted from the set bed area 40, the image processing system 300 determines that the setting relating to the action determination process has become incorrect, and displays the abnormal time screen 360.

  In one aspect, the image processing system 300 notifies the administrator by means of a message 50 that the settings relating to the action determination process are not correct. In another aspect, the image processing system 300 notifies the administrator by a sound such as a voice that the setting related to the action determination process is not correct. As a result, the image processing system 300 can prompt the administrator to make settings related to the action determination process.

[Functional Configuration of Image Processing System 300]
The function of the image processing system 300 will be described with reference to FIG. FIG. 10 is a block diagram illustrating an example of a functional configuration of the image processing system 300. As shown in FIG. 10, the image processing system 300 includes an indoor terminal 100 and a management server 200. Below, the function of the indoor terminal 100 and the management server 200 is demonstrated in order.

(Functional configuration of indoor terminal 100)
As illustrated in FIG. 10, the indoor terminal 100 includes a reception unit 110, a person detection unit 120, a behavior determination unit 130, a deviation detection unit 150, and a transmission unit 160 as functional configurations.

  The receiving unit 110 receives the bed area 40 set for the setting image 30 </ b> A (FIG. 2) from the management server 200. The reception unit 110 outputs the received bed area 40 to the behavior determination unit 130 and the deviation detection unit 150.

  The person detection unit 120 detects a person from an input image 30B (FIG. 2) obtained by photographing a subject from the same viewpoint as the setting image 30A. Details of the person detection process will be described later (see FIG. 17).

  The behavior determination unit 130 determines the behavior of the person with respect to the bed based on the positional relationship between the bed region 40 in the input image 30B and the person region in the input image 30B. Details of the action determination process will be described later (see FIG. 16).

  The shift detection unit 150 compares the image information of the bed area 40 in the setting image 30A and the image information of the bed area 40 in the input image 30B, or inputs the image information in the vicinity of the bed area 40 in the setting image 30A. The image information in the vicinity of the bed area 40 in the image 30B is compared to detect that the bed is displaced from the bed area 40. Details of the processing for detecting bed displacement will be described later (see FIGS. 14 and 15).

  The transmission unit 160 informs the management server 200 of the setting image 30A obtained by the camera 105 (FIG. 20), the type of behavior determined by the behavior determination unit 130, and the deviation of the object detected by the deviation detection unit 150. Send.

(Functional configuration of management server 200)
The functional configuration of the management server 200 will be described with reference to FIG. As illustrated in FIG. 10, the management server 200 includes a reception unit 210, a deviation notification unit 220, a behavior notification unit 230, a reception unit 240, and a transmission unit 250 as functional configurations.

  The receiving unit 210 transmits from the indoor terminal 100 the setting image 30A obtained from the camera 105 (FIG. 20), the type of action determined by the action determining unit 130, and the deviation of the object detected by the deviation detecting unit 150. Receive.

  The shift notification unit 220 notifies that the bed is shifted from the set bed area 40. As an example of the notification unit, the shift notification unit 220 displays a message that the bed is shifted from the bed region 40 as a message or outputs it by voice.

  The behavior notification unit 230 notifies that the behavior is detected when a predetermined behavior of the person on the bed is detected. The predetermined action to be notified includes, for example, a person getting up, getting out of bed, and other actions that are dangerous for the person to be monitored. As an example of notification means, the behavior notification unit 230 displays information indicating the type of behavior as a message or outputs it by voice.

  The accepting unit 240 accepts the setting of the bed area 40 including the object with respect to the setting image 30A (FIG. 2) obtained by photographing the subject. As an example, the receiving unit 240 receives the setting of the bed region 40 by receiving at least one of a plurality of point settings for the setting image 30A, a plurality of line settings for the setting image 30A, and a surface setting for the setting image 30A. Accept. The accepting unit 240 may further accept the setting of the bed orientation in the setting image 30A.

  The transmission unit 250 transmits the setting of the bed area 40 for the setting image 30 </ b> A received by the reception unit 240 to the indoor terminal 100.

[Pre-processing of Image Processing System 300]
With reference to FIG. 11 and FIG. 12, the preliminary process performed before starting an action discrimination | determination process is demonstrated. FIG. 11 is a flowchart illustrating a pre-process that is executed before the behavior determination process is started. The process of FIG. 11 is executed by, for example, a CPU (Central Processing Unit) 202 of the management server 200 (FIG. 20). In other aspects, some or all of the processing may be performed by circuit elements or other hardware. FIG. 12 is a conceptual diagram schematically showing pre-processing executed before starting the behavior determination processing.

  In step S10, the image processing system 300 receives the setting of the bed area 40 for the setting image 30A as the above-described receiving unit 240 (FIG. 10). As an example, the image processing system 300 receives input of points 41 </ b> A to 41 </ b> D corresponding to the four ends of the bed 20, and sets a region surrounded by the points 41 </ b> A to 41 </ b> D as the bed region 40.

  In step S <b> 12, the image processing system 300 calculates a luminance value distribution (hereinafter also referred to as “histogram”) around the four corners of the set bed area 40 for each corner of the bed area 40. FIG. 12 shows an example in which the histogram 61 is calculated from the luminance values around the point 41A among the points 41A to 41D. For example, the histogram 61 is calculated from luminance values of 5 × 5 pixels around the point 41A. More specifically, the image processing system 300 calculates the number of pixels having the same luminance value for each luminance value in the 5 × 5 pixel region.

  In step S14, the image processing system 300 stores the four histograms calculated for each of the four corners of the bed area 40 as the template 106A (FIG. 20) in the storage device 206 (FIG. 20). The template 106A is used in a deviation detection process described later.

[Main Processing of Image Processing System 300]
(Outline of this process)
With reference to FIGS. 13 to 19, the main process of the image processing system 300 executed after the above-described pre-process will be described. FIG. 13 is a flowchart showing this processing of the image processing system 300. The processing in FIG. 13 is executed by the CPU 102 (FIG. 20) of the indoor terminal 100 and the CPU 202 (FIG. 20) of the management server 200. In other aspects, some or all of the processing may be performed by circuit elements or other hardware.

  In step S40, the image processing system 300 acquires an input image obtained by photographing a person to be monitored (for example, a cared person) from the camera 105 (FIG. 20).

  In step S <b> 50, the image processing system 300 determines whether or not there is an abnormality in the setting related to the action determination process (for example, the setting of the bed area 40) set in the above-described pre-processing as the above-described deviation detection unit 150 (FIG. 10). Judging. Details of the abnormality detection process will be described later (see FIGS. 14 and 15).

  In step S52, when the image processing system 300 determines that there is an abnormality in the setting relating to the action determination process (YES in step S52), the control is switched to step S62. Otherwise (NO in step S52), image processing system 300 switches control to step S60.

  In step S60, the image processing system 300 performs a behavior determination process for the care receiver as the behavior determination unit 130 described above (FIG. 10). Details of the action determination process will be described later (see FIG. 16).

  In step S62, image processing system 300 determines whether or not to end image processing according to the present embodiment. For example, image processing system 300 determines that the image processing according to the present embodiment is to be terminated when an operation for interrupting the processing is received from the administrator (YES in step S62). Otherwise (NO in step S62), image processing system 300 switches control to step S64.

  In step S <b> 64, the image processing system 300 acquires the next input image from the camera 105. As a result, the image processing system 300 sequentially executes image processing according to the present embodiment on the time-series images.

  Hereinafter, details of the abnormality detection process in step S50 and the action determination process in step S60 will be described in order.

(A) Details of Abnormality Detection Processing With reference to FIGS. 14 and 15, the abnormality detection processing of the image processing system 300 will be described in detail. FIG. 14 is a flowchart showing the abnormality detection process executed in step S50 of FIG. The processing in FIG. 14 is executed by the CPU 102 (FIG. 20) of the indoor terminal 100 and the CPU 202 (FIG. 20) of the management server 200. FIG. 15 is a conceptual diagram schematically showing the abnormality detection process.

  As described above, the image processing system 300 detects that the content set for the behavior determination process is not normal by the abnormality detection process.

  More specifically, in step S102, the image processing system 300 sets the bed area 40 of the input image 30B (FIG. 2), which is an area corresponding to the bed area 40 set for the setting image 30A (FIG. 2). A luminance value distribution (that is, a histogram) at each corner is calculated from around the four corners.

  In step S104, the image processing system 300 compares histograms around corners corresponding to each other in the bed area 40 in the setting image 30A and the bed area 40 in the input image 30B. FIG. 15 shows an example in which a histogram 61 around the upper left corner of the bed area 40 in the setting image 30A is compared with a histogram 62 around the upper left corner of the bed area 40 in the input image 30B. The image processing system 300 subtracts one of the histograms 61 and 62 from the other histogram. Thereafter, the image processing system 300 calculates an absolute value sum of the difference results. The larger the absolute value sum is, the more the image information of the bed area 40 in the setting image 30A and the image information of the bed area 40 in the input image 30B are different. That is, this indicates that the bed is displaced from the preset bed area 40.

  In step S104, when image processing system 300 determines that the sum of absolute values is empirically larger than a preset threshold value (YES in step S104), control is switched to step S120. If not (NO in step S104), image processing system 300 switches control to step S110.

  In step S110, the image processing system 300 determines that the setting content related to the action determination process is “normal”. That is, the image processing system 300 determines that the bed is not displaced from the preset bed area 40.

  In step S120, the image processing system 300 determines that the setting content related to the action determination process is “abnormal”. That is, the image processing system 300 determines that the bed is shifted from the bed area 40 set in advance. In step S122, the image processing system 300 notifies the administrator that the bed has shifted from the preset bed area 40 as the above-described shift notification unit 220 (FIG. 10).

  Note that the method of detecting bed displacement is not limited to the example shown in FIGS. 14 and 15. For example, the image processing system 300 extracts and extracts a feature amount indicating a characteristic portion (for example, an edge portion) of the bed from both the bed region 40 of the setting image 30A and the bed region 40 of the input image 30B. You may detect the shift | offset | difference of a bed by comparing two feature-values.

  Further, the image processing system 300 may detect the bed shift by comparing the orientation of the bed in the bed area 40 in the setting image 30A and the orientation of the bed in the bed area 40 in the input image 30B. The image processing system 300 determines that the bed is displaced when these directions differ by a predetermined angle or more.

  Furthermore, the image processing system 300 may detect bed displacement by a method other than image processing. For example, the image processing system 300 detects bed displacement using an output from a distance sensor (not shown). More specifically, the distance sensor is installed around the bed and measures the distance between itself and the bed. The image processing system 300 determines that the bed is displaced based on the fact that the distance is not included in the preset distance range. In addition, the image processing system 300 may detect the displacement of the bed using a sensor such as a thermography or another sensor that can detect the position of the bed. Further, the image processing system 300 may detect the bed shift by combining the detection result of the bed shift by such a sensor and the detection result of the bed shift by the image processing described above.

  Furthermore, the setting content of the behavior determination process that is an abnormality detection target is not limited to the shift of the bed region 40. The image processing system 300 can determine an abnormality with respect to any setting content related to the action determination process of the person 12. For example, when accepting the setting of an area (hereinafter also referred to as “excluded area”) that the image processing system 300 does not use for the action determination process, the image processing system 300 detects the displacement of the excluded area and detects the excluded area. It may be configured to notify the administrator that there is a deviation.

(B) Details of Behavior Determination Processing With reference to FIGS. 16 to 19, the behavior determination processing of the image processing system 300 will be described in detail. FIG. 16 is a flowchart showing the action determination process executed in step S60 of FIG.

  In step S152, the image processing system 300 detects a person region from the input image 30B (FIG. 2) as the person detection unit 120 described above (FIG. 10). Details of the person area detection processing will be described later (see FIG. 17).

  In step S154, the image processing system 300 causes the person shown in the input image 30B to wake up based on the relative positional relationship between the bed area 40 in the input image 30B and the person area in the input image 30B. It is determined whether or not. Details of the wake-up discrimination process will be described later (see FIG. 18).

  In step S156, the image processing system 300 determines that the person shown in the input image 30B has left the bed based on the relative positional relationship between the bed area 40 in the input image 30B and the person area in the input image 30B. It is determined whether or not. Details of the wake-up determination process will be described later (see FIG. 19).

  Hereinafter, details of the person area extraction process in step S152, the wake-up determination process in S154, and the wake-up determination process in S156 will be described in order.

(B-1) Person Area Extraction Processing A process for extracting a person area from the input image 30B will be described with reference to FIG. FIG. 17 is a flowchart showing an outline of the person area extraction (detection) process executed in step S152 of FIG.

  The person region extraction process is realized by, for example, a background difference that makes a difference between background images from an input image, or a time difference that makes an difference between images whose shooting times are before and after. FIG. 17 schematically shows a process of extracting the person region 70 from the input image 30B based on the background difference.

  More specifically, the image processing system 300 acquires a background image 30C in which no person is shown in advance. The background image 30C may be the same image as the setting image 30A (FIG. 2) described above, or may be an image obtained separately from the setting image 30A.

  When the image processing system 300 acquires the input image 30B from the camera 105 (FIG. 20), the image processing system 300 subtracts the background image 30C from the input image 30B. Thereby, the image processing system 300 can remove the background from the input image 30B. The image processing system 300 sets a region having a pixel value equal to or greater than a predetermined value as the person region 70.

  The person area 70 may be extracted by a method different from the method shown in FIG. For example, the image processing system 300 prepares in advance a characteristic portion (that is, a feature amount) of a person as a template, and searches for an area similar to the template by scanning the input image 30B. When the image processing system 300 can search for an area similar to the template in the input image 30B, the image processing system 300 sets the area as the person area 70. In addition, the person region 70 may be extracted by other image processing techniques such as optical flow and tracking.

(B-2) Wake-up Discrimination Processing With reference to FIG. 18, the wake-up discrimination processing of the image processing system 300 will be described in detail. FIG. 18 is a flowchart showing the wake-up determination process executed in step S154 of FIG.

  In step S202, the image processing system 300 determines whether or not the current action state of the person 12 (FIG. 17) shown in the input image 30B is “before waking up”. If the image processing system 300 determines that the current action state of the person is “before waking up” (YES in step S202), the image processing system 300 switches the control to step S204. If not (NO in step S202), the image processing system 300 ends the wakeup determination process.

  In step S204, the image processing system 300 determines the input image based on at least one of the size of the person area S1 (FIG. 17) occupying outside the bed area and the size of the person area S2 (FIG. 17) occupying the bed area. The action of the person 12 shown in 30B is determined. For example, the image processing system 300 determines whether or not the number of pixels of the person area S1 outside the bed area is larger than a threshold value Th1 set in advance by experience. If the image processing system 300 determines that the number of pixels in the person area S1 outside the bed area is larger than the threshold Th1 (YES in step S204), the control is switched to step S220. Otherwise (NO in step S204), image processing system 300 switches control to step S210.

  In step S210, the image processing system 300 sets the current behavior state of the person 12 in the input image 30B to “before getting up”.

  In step S <b> 220, the image processing system 300 determines that the person 12 shown in the input image 30 </ b> B has woken up and notifies the person 12 of the wakeup. In step S222, the image processing system 300 sets the current behavior state of the person 12 in the input image 30B to “after waking up”.

  Thus, the image processing system 300 determines that the person 12 has woken up based on the person area S1 occupying outside the bed area 40 of the input image 30B exceeding the threshold Th1. That is, the image processing system 300 calculates the size of the person region S1 that occupies outside the bed region 40 of the input image 30B from the input image 30B in chronological order, and the size of the sequentially calculated person region S1 is greater than the threshold Th1. It is determined that the person 12 has woken up on the basis of the increase. In other words, in the image processing system 300, when the size of the previous person area S1 is smaller than the threshold value Th1, and the size of the current person area S1 is larger than the threshold value Th1, the person 12 is awakened. Determine.

  Alternatively, the image processing system 300 calculates the size of the person area S2 occupying the bed area 40 of the input image 30B from the input image 30B in chronological order, and the size of the person area S2 calculated sequentially is greater than a predetermined threshold. May be determined that the person 12 has woken up. In other words, in the image processing system 300, when the size of the previous person area S2 is larger than the predetermined threshold and the size of the current person area S2 is smaller than the predetermined threshold, the person 12 gets up. Determine.

  In the above description, the example in which the image processing system 300 determines the rising of the person 12 using two or more input images 30B has been described. However, the image processing system 300 uses one input image 30B. The rising of the person 12 may be determined. In this case, the image processing system 300 determines that the person 12 has woken up when the person area S2 exists on the edge of the bed area 40 of the input image 30B. For example, when the person area S1 outside the bed area 40 of the input image 30B is larger than a predetermined threshold and the person area S2 in the bed area 40 of the input image 30B is larger than the predetermined threshold, the image processing system 300 It is determined that 12 has woken up. Further, the image processing system 300 may determine the wakeup of the person 12 using the ratio of the person area S1 and the person area S2.

(B-3) Bed leaving determination process With reference to FIG. 19, the bed leaving determination process of the image processing system 300 will be described in detail. FIG. 19 is a flowchart showing the bed leaving determination process executed in step S156 of FIG.

  In step S252, the image processing system 300 determines whether or not the current action state of the person 12 (FIG. 17) shown in the input image 30B is “before leaving the bed”. If the image processing system 300 determines that the current action state of the person is “before getting out of bed” (YES in step S252), the control is switched to step S254. If not (NO in step S252), the image processing system 300 ends the bed leaving determination process.

  In step S254, the image processing system 300 determines the input image based on at least one of the size of the person area S1 (FIG. 17) occupying outside the bed area and the size of the person area S2 (FIG. 17) occupying the bed area. The action of the person 12 shown in 30B is determined. For example, the image processing system 300 determines whether or not the number of pixels of the person area S2 in the bed area is smaller than a threshold value Th2 that is set empirically in advance. When the image processing system 300 determines that the number of pixels of the person area S2 in the bed area is smaller than the threshold Th2 (YES in step S254), the control is switched to step S270. Otherwise (NO in step S254), image processing system 300 switches control to step S260.

  In step S260, the image processing system 300 sets the current behavior state of the person 12 shown in the input image 30B to “before leaving”.

  In step S <b> 270, the image processing system 300 determines that the person 12 shown in the input image 30 </ b> B has left the floor and notifies the person 12 of getting out of bed. In step S272, the image processing system 300 sets the current behavior state of the person 12 shown in the input image 30B to “after leaving the bed”.

  In this way, the image processing system 300 determines that the person 12 has left the floor based on the fact that the person area S2 occupying the bed area 40 of the input image 30B has fallen below the threshold Th2. That is, the image processing system 300 calculates the size of the person area S2 in the bed area 40 of the input image 30B from the input image 30B in chronological order, and the size of the person area S2 is smaller than the threshold Th2. Based on this, it is determined that the person 12 has left the floor.

  Alternatively, the image processing system 300 calculates the size of the person area S1 occupying outside the bed area 40 of the input image 30B from the input image 30B in chronological order, and the size of the person area S1 becomes larger than the threshold Th2. Based on this, it may be determined that the person 12 has left the floor.

  In the above description, the example in which the image processing system 300 determines whether the person 12 is getting out of bed using two or more input images 30B has been described. However, the image processing system 300 uses one input image 30B. You may determine whether the person 12 is getting out of bed. In this case, for example, the image processing system 300 determines whether the person 12 is getting out of bed when the entire person area exists outside the bed area 40. Further, the image processing system 300 may determine whether the person 12 is getting out of bed using the ratio of the person area S1 occupying outside the bed area 40 and the person area S2 occupying the bed area 40.

[Hardware Configuration of Image Processing System 300]
An example of the hardware configuration of the image processing system 300 will be described with reference to FIG. FIG. 20 is a block diagram illustrating a main hardware configuration of the image processing system 300. As illustrated in FIG. 20, the image processing system 300 includes an indoor terminal 100, a management server 200, and a network 400. The indoor terminal 100 and the management server 200 are connected via a network 400. Hereinafter, the hardware configuration of the indoor terminal 100 and the hardware configuration of the management server 200 will be described in order.

(Hardware configuration of indoor terminal 100)
As shown in FIG. 20, the indoor terminal 100 includes a ROM (Read Only Memory) 101, a CPU 102, a RAM (Random Access Memory) 103, a network I / F (interface) 104, a camera 105, and a storage device. 106.

  The ROM 101 stores an operating system (OS), a control program executed by the indoor terminal 100, and the like. The CPU 102 controls the operation of the indoor terminal 100 by executing various programs such as an operating system and a control program for the indoor terminal 100. The RAM 103 functions as a working memory and temporarily stores various data necessary for program execution.

  An information transmission / reception device such as an antenna or a NIC (Network Interface Card) is connected to the network I / F 104. The indoor terminal 100 transmits / receives data to / from other communication devices via the information transmitting / receiving device. Other communication devices include, for example, the management server 200 and other communication devices. Indoor terminal 100 may be configured to download a program for realizing various processes according to the present embodiment via network 400.

  A camera 105, for example, a surveillance camera or other imaging device capable of photographing a subject. Note that the camera 105 may be configured separately from the indoor terminal 100, or may be configured integrally with the indoor terminal 100 as shown in FIG.

  The storage device 106 is a storage medium such as a hard disk or an external storage device. As an example, the storage device 106 stores a bed area 40, a template 106A, and an image processing program 106B for realizing various processes according to the present embodiment.

  The bed area 40 is information for specifying an area in which a bed is shown in a setting image or an input image. For example, the bed area 40 is indicated by coordinate values indicating the four corners of the bed.

  The template 106A corresponds to, for example, four histograms calculated for each of the four corners around the bed area 40 (see FIG. 12). The bed area 40, the template 106A, and the image processing program 106B may be stored not in the storage device 106 but in the indoor terminal 100 or a server.

  Note that the image processing program 106B may be provided by being incorporated in a part of an arbitrary program, not as a single program. In this case, processing according to the present embodiment is realized in cooperation with an arbitrary program. Even such a program that does not include some modules does not depart from the spirit of the image processing system 300 according to the present embodiment. Furthermore, part or all of the functions provided by image processing program 106B according to the present embodiment may be realized by dedicated hardware. Furthermore, the indoor terminal 100 and the management server 200 may cooperate to realize processing according to the present embodiment. Furthermore, the management server 200 may be configured in the form of a so-called cloud service in which at least one server realizes processing according to the present embodiment.

(Hardware configuration of management server 200)
Next, the hardware configuration of the management server 200 will be described. As illustrated in FIG. 20, the management server 200 includes a ROM 201, a CPU 202, a RAM 203, a network I / F 204, a monitor 205, and a storage device 206.

  The ROM 201 stores an operating system, a control program executed by the management server 200, and the like. The CPU 202 controls the operation of the management server 200 by executing various programs such as an operating system and a control program of the management server 200. The RAM 203 functions as a working memory and temporarily stores various data necessary for program execution.

  An information transmission / reception device such as an antenna or NIC is connected to the network I / F 204. The management server 200 transmits / receives data to / from other communication devices via the information transmitting / receiving device. Other communication devices include, for example, the indoor terminal 100 and other communication devices. Management server 200 may be configured to download a program for realizing various processes according to the present embodiment via network 400.

  The monitor 205 includes a main screen 310 (FIG. 4), a setting mode top screen 320 (FIG. 5), an area setting screen 330 (FIG. 6), a normal screen 340 (FIG. 7), a notification time screen 350 (FIG. 8), Further, a screen such as an abnormal time screen 360 (FIG. 9) is displayed. The monitor 205 may be realized as a touch panel in combination with a touch sensor (not shown). The touch panel receives the above-described bed area setting, screen switching operation, and the like by a touch operation.

  The storage device 206 is a storage medium such as a hard disk or an external storage device. As an example, the storage device 206 stores an image processing program for realizing various processes according to the present embodiment.

[Brief Summary]
As described above, the image processing system 300 according to the present embodiment automatically detects the shift of the bed 20 when the bed shifts from the initially set bed area 40, and the administrator detects that the bed has shifted. To inform. Since the bed area 40 set in advance is used for the action determination process of the person to be monitored, the displacement of the bed greatly affects the accuracy of the action determination process. The image processing system 300 can prompt the administrator to correct the setting of the bed area by notifying the administrator that the bed is displaced. As a result, the image processing system 300 can prevent the accuracy of the action determination process from being lowered, and can reduce the burden on the caregiver by being notified of getting up or getting out of bed by mistake.

<Second Embodiment>
[Overview]
The image processing system 300 according to the first embodiment always reports the displacement when the bed is displaced from the bed region 40. On the other hand, even if the image processing system 300A according to the second embodiment is a case where the bed is displaced from the bed region 40, the displacement may not be notified. For example, when the following conditions (A) to (C) are satisfied, the image processing system 300A does not notify that the bed is displaced.

(A) When it is set not to notify the displacement When the image processing system 300A is set by the administrator so as not to notify the displacement even when the bed is displaced from the bed area 40 , The deviation is not notified.

(B) When the installation condition of the camera 105 satisfies a predetermined prohibition condition When the installation condition of the camera 105 is changed, it may be determined that the bed has shifted even though the bed has not shifted. Therefore, even when the bed is displaced from the bed area 40, the image processing system 300A does not notify the displacement when the installation conditions of the camera 105 satisfy a predetermined prohibition condition. As an example, the prohibition condition is satisfied when the direction of the camera 105 deviates from a preset direction due to maintenance of the camera 105 or the like.

(C) When the environment shown in the input image satisfies a predetermined condition The image processing system 300A satisfies the predetermined condition even if the bed is displaced from the bed area 40. When this occurs, the deviation is not notified.

  As an example, when the lighting environment around the camera 105 changes, the image information of the input image changes as a whole, and therefore it may be determined that the bed has shifted even though the bed has not shifted. Therefore, when the image processing system 300A determines that the lighting environment around the camera 105 has changed, the image processing system 300A does not notify the bed shift. More specifically, the image processing system 300A sequentially calculates the brightness around the camera 105 from the image information of the input image, and when the current brightness changes by a predetermined value or more than the previous brightness, the illumination condition Is determined to have changed. The brightness is indicated by an average value of pixel values of the input image, for example.

  In addition, when a family member or a caregiver is near a cared person, the need to monitor the behavior of the cared person is low. Therefore, when two or more persons are detected from the input image, the image processing system 300A determines that the environment indicated in the input image satisfies a predetermined condition and does not notify the bed shift.

  Furthermore, when the care recipient is removed from the field of view of the camera 105, the behavior of the care recipient cannot be monitored, so it does not make sense to monitor the behavior of the care recipient. Therefore, when the care receiver is away from the bed by a predetermined distance, the image processing system 300A determines that the environment indicated in the input image satisfies a predetermined condition and does not notify the bed shift. For example, the distance between the care receiver and the bed is calculated by the above-described person area extraction process.

  Alternatively, the distance may be calculated using a tag such as an RFID (Radio Frequency Identifier). The RFID holds ID (Identification) information and exchanges information with the reader by short-range wireless communication using an electromagnetic field or a radio wave. The reader is provided in the bed or around the bed. For example, when the reader is within a range of several centimeters to several meters from itself, the reader can exchange information with the RFID. That is, when the care receiver holding the RFID is within a predetermined distance from the reader, the reader can exchange information with the RFID. On the other hand, the reader cannot exchange information with the RFID when the cared person holding the RFID is separated from the reader by a predetermined distance. By utilizing this, the image processing system 300A determines that the care receiver is separated from the bed by a predetermined distance when the reader cannot exchange information with the RFID.

[Variation 1 of Abnormality Detection Processing]
A modification of the abnormality detection process shown in FIG. 14 will be described with reference to FIG. FIG. 21 is a flowchart showing a modification of the abnormality detection process shown in FIG. Of the steps shown in FIG. 21, the processes other than step S250 are the same as the steps shown in FIG. 14, so description thereof will not be repeated.

  As described above, the image processing system 300 </ b> A may not notify the displacement even when the bed is displaced from the bed region 40. Whether or not to notify the deviation is determined in step S250 of FIG.

  More specifically, in step S250, image processing system 300A determines whether or not to notify that the bed is displaced. For example, the image processing system 300A determines not to notify that the bed is displaced when the above conditions (A) to (C) are satisfied. When image processing system 300A determines to notify that the bed is displaced (YES in step S250), control is switched to step S122. Otherwise (NO in step S250), image processing system 300A ends the abnormality detection process.

  The time for prohibiting the notification of the deviation is arbitrary. For example, the image processing system 300A prohibits the notification of the deviation only for a predetermined time after the above conditions (A) to (C) are satisfied. Alternatively, the image processing system 300A may prohibit the notification of deviation until the above-described conditions (A) to (C) are satisfied and an operation for canceling the prohibition of notification is received from the administrator. Alternatively, the image processing system 300A may prohibit the notification of deviation until the conditions (A) to (C) are not satisfied after the conditions (A) to (C) are satisfied.

[Variation 2 of abnormality detection processing]
Hereinafter, Modification 2 of the abnormality detection process will be described. In the first modification of the abnormality detection process, the image processing system 300A does not notify the bed shift when the above conditions (A) to (C) are satisfied. On the other hand, in the present modification, the image processing system 300 </ b> A has a condition for detecting a bed shift more than the currently set condition when the above conditions (A) to (C) are satisfied. Make it strict.

  For example, when the above conditions (A) to (C) are satisfied, the image processing system 300A increases the predetermined threshold value shown in step S104 of FIG. 14 from the current value. Thereby, it becomes difficult to determine that the bed is displaced from the bed region 40, and erroneous reporting of the bed displacement is suppressed.

[Brief Summary]
As described above, the image processing system 300A according to the second embodiment does not notify that the bed is shifted when the above conditions (A) to (C) are satisfied. As a result, the image processing system 300A can prevent the bed shift from being notified even though the bed is not shifted.

  The embodiment disclosed this time should be considered as illustrative in all points and not restrictive. The scope of the present invention is defined by the terms of the claims, rather than the description above, and is intended to include any modifications within the scope and meaning equivalent to the terms of the claims.

  10 care recipients, 12 persons, 20 beds, 30A setting image, 30B input image, 30C background image, 40 bed area, 41A to 41D points, 50, 352 message, 61, 62 histogram, 70, S1, S2 person area, 100 indoor terminal, 101,201 ROM, 102,202 CPU, 103,203 RAM, 104,204 network I / F, 105 camera, 106,206 storage device, 106A template, 106B image processing program, 110,210 receiving unit, 120 person detection unit, 130 behavior determination unit, 150 deviation detection unit, 160, 250 transmission unit, 200 management server, 205 monitor, 220 deviation notification unit, 230 behavior notification unit, 240 reception unit, 300, 300A image processing system, 310 Maine Screen, 312, 314, 322, 324 button, 320 setting mode top screen, 330 area setting screen, 332 pointer, 340 normal screen, 350 alerting screen, 360 abnormal screen, 400 network, Th1, Th2 threshold.

Claims (9)

An accepting unit for accepting a setting of an area including an object with respect to a reference image obtained by photographing a subject;
A person detection unit for detecting a person region from an input image obtained by photographing the subject from the same viewpoint as the reference image;
An action determination unit for determining an action of a person in the person area based on a positional relationship between the area including the object in the input image and the person area in the input image;
The image information of the area including the object in the reference image is compared with the image information of the area including the object in the input image, or the image information near the area including the object in the reference image and the information A displacement detection unit for comparing the image information in the vicinity of the region including the object in the input image and detecting that the object is displaced from the region including the object in the input image ;
When the displacement of the object is detected in the input image, the fact that the object in the input image is displaced from the set region including the object is notified without determining the action of the person. An image processing system.   The receiving unit receives a setting of a region including the object by receiving at least one of a plurality of point settings for the reference image, a plurality of line settings for the reference image, and a surface setting for the reference image. The image processing system according to claim 1.   The image processing system according to claim 2, wherein the reception unit further receives a setting of an orientation of the object in the reference image.   The behavior determination unit is configured to determine the person based on at least one of a size of a person area that occupies an area that includes the object in the input image and a size of a person area that occupies an area outside the area that includes the object in the input image. The image processing system according to claim 1, wherein the behavior is discriminated. The object is bedding;
The person detection unit sequentially detects a person region from a plurality of time-series input images,
The behavior determination unit
Sequentially calculating the size of the person area outside the area including the object of the input image;
The image processing system according to claim 4, wherein the image processing system determines that the person has woken up based on the sequentially calculated size of the person area being greater than a predetermined threshold. The object is bedding;
The person detection unit sequentially detects a person region from a plurality of time-series input images,
The behavior determination unit
Sequentially calculating the size of the person area in the area including the object of the input image;
The image processing system according to claim 4, wherein it is determined that the person has left the floor based on the sequentially calculated size of the person area being smaller than a predetermined threshold. An accepting unit for accepting a setting of an area including an object with respect to a reference image obtained by photographing a subject;
A person detection unit for detecting a person region from an input image obtained by photographing the subject from the same viewpoint as the reference image;
An action determination unit for determining an action of a person in the person area based on a positional relationship between the area including the object in the input image and the person area in the input image;
The image information of the area including the object in the reference image is compared with the image information of the area including the object in the input image, or the image information near the area including the object in the reference image and the information A displacement detection unit for comparing the image information in the vicinity of the region including the object in the input image and detecting that the object is displaced from the region including the object in the input image ;
When the displacement of the object is detected in the input image, the fact that the object in the input image is displaced from the set region including the object is notified without determining the action of the person. An image processing device. Receiving a setting of a region including an object with respect to a reference image obtained by photographing a subject;
Detecting a person region from an input image obtained by photographing the subject from the same viewpoint as the reference image;
Determining a behavior of a person in the person area based on a positional relationship between the area including the object in the input image and the person area in the input image;
The image information of the area including the object in the reference image is compared with the image information of the area including the object in the input image, or the image information near the area including the object in the reference image and the information Comparing image information in the vicinity of the region including the object in the input image and detecting that the object is shifted from the region including the object in the input image ;
When the displacement of the object is detected within the input image, the step of notifying that without determine the behavior of the person, are out of the object in the input image from the area including the object that has been set An image processing method comprising: An image processing program,
The image processing program is stored in a computer.
Receiving a setting of a region including an object with respect to a reference image obtained by photographing a subject;
Detecting a person region from an input image obtained by photographing the subject from the same viewpoint as the reference image;
Determining a behavior of a person in the person area based on a positional relationship between the area including the object in the input image and the person area in the input image;
The image information of the area including the object in the reference image is compared with the image information of the area including the object in the input image, or the image information near the area including the object in the reference image and the information Comparing image information in the vicinity of the region including the object in the input image and detecting that the object is shifted from the region including the object in the input image ;
When the displacement of the object is detected within the input image, the step of notifying that without determine the behavior of the person, are out of the object in the input image from the area including the object that has been set An image processing program for executing

Images