JP6917294B2 - Watching system - Google Patents

Description

The present invention relates to a watching system.

There is a system that monitors elderly people, sick and injured people, children, pet animals, etc. from a distance by using images taken by a camera and data acquired by various sensors. In Patent Document 1 below, an image taken by a camera is captured by a home device and stored in a memory, and when a person is detected by a human body sensor, the sensor reaction is used as a trigger to call the image stored in the memory. A remote monitoring system that transmits to a server device via a line and the Internet and further transmits an image from the server device to a mobile terminal is disclosed.

Japanese Unexamined Patent Publication No. 2005-316883

For the person being watched over (the person to be watched over), it is psychologically reluctant to convey the situation of the person being watched over in the real space such as an image taken by the camera to the person who is watching over (the person being watched over) as it is. .. In addition, it is assumed that it is a great psychological burden for the watcher to continuously watch the situation of the watchee in the real space.

The present invention reduces psychological resistance and makes the watcher feel warmer than the case where the watcher and / or the watch target person are watched by displaying the situation in the real space as it is. The purpose is to provide a watching service that can be done.

The present invention that achieves the above object is a grasping means for grasping the situation of a watching target person, a background image holding means for holding a background image that replaces the background of the watching target person, and a character that replaces the watching target person. It is provided with a character image holding means for holding an image and a display means for expressing the situation of the watching target person grasped by the grasping means by a character image arranged on a background image, and the situation of the watching target person is a reality. includes context background watching target person in space, in replacement of the background image of the background of the watching target person, contents and location of goods and equipment that are present in the real space is considered, characterized in Rukoto, Protection system Is.
Here, the display means may change the state of the displayed character according to the situation of the watching target person grasped by the grasping means.
Further, the display means may display a predetermined character-like effect for each displayed character, in addition to the change in the state of the character according to the situation of the watching target person grasped by the grasping means. good.
Further, the present invention further includes a background selection receiving means for accepting the selection of a background image to replace the background of the person to be watched in the above-mentioned watching system, and the display means is the background image held by the background image holding means. Of these, the background image set according to the selection result received by the background selection receiving means may be displayed.
Further, the present invention further includes a character selection receiving means for receiving the selection of a character image to be a substitute for the watching target person in the above-mentioned watching system, and the display means is among the character images held by the character image holding means. , The character image set according to the selection result received by the character selection receiving means may be displayed.
In addition, another invention that achieves the above object includes an acquisition means for acquiring information on a person to be watched, a background image holding means for holding a background image that replaces the background of the person to be watched, and a means for holding a person to be watched. Based on the character image holding means that holds the character image to be replaced and the information of the watching target person acquired by the acquisition means, the background image and the character image are combined to generate an image showing the situation of the watching target person. The situation of the person to be watched includes the situation of the background of the person to be watched in the real space, and the goods and equipment existing in the real space are replaced with the background image of the background of the person to be watched. content and position are considered, characterized in Rukoto a watch system.

According to the present invention, the psychological resistance is reduced and the psychological resistance is reduced as compared with the case where the photographed image of the real space is displayed as it is for the watching person and / or one of the watching person. It is possible to provide a watching service that makes you feel warmer.

It is a figure which shows the whole structure of the monitoring system by this embodiment. It is a figure which shows the structure of the subject system. It is a figure which shows the functional structure of a center server. It is a figure which shows the functional structure of a watcher terminal. 5 (a) is a diagram showing a display example of a content image, FIG. 5 (a) is a diagram showing a state in which a captured image of a person to be watched is displayed as it is, and FIG. It is a figure which shows the state which displayed the content image generated based on. It is a figure which shows an example of the content image composed by a specific theme. It is a figure which shows the example of the display mode of the content image. It is a figure which shows the structure of the watcher terminal at the time of bidirectional information transmission. It is a figure which shows the functional structure of the subject system at the time of bidirectional information transmission. It is a figure which shows the display example of the content image of a watcher terminal and a target person system in the case of bidirectional information transmission.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.
<System configuration>
FIG. 1 is a diagram showing an overall configuration of a monitoring system according to the present embodiment. The watching system is a system that realizes a watching service for a person to be watched and a watcher. The watching system shown in FIG. 1 includes a target person system 100, a center server 200, and a watching person terminal 300. Each of the target person system 100, the center server 200, and the watcher terminal 300 is connected via the network 400.

The target person system 100 is a system for acquiring information on a monitoring target person. The subject system 100 is composed of a plurality of device groups. The center server 200 is a server that processes and processes the information of the watching target person obtained by the target person system 100 and provides it to the watching person terminal 300. In the example shown in FIG. 1, one center server 200 is shown, but the center server 200 is not limited to a single server machine, and the center server 200 is distributed to a plurality of server machines provided on the network. Functions may be realized (so-called cloud environment, etc.). The watcher terminal 300 is a terminal device that presents the information sent from the center server 200 to the watcher. The watcher terminal 300 may be configured as a single device or may be configured by a plurality of devices. The network 400 is not particularly limited as long as it is a communication network used for data communication between each system and device, and may be, for example, a LAN (Local Area Network), a WAN (Wide Area Network), the Internet, or the like. The communication line used for data communication may be a combination of these, regardless of whether it is wired or wireless. Further, a relay device such as a gateway device or a router may be used to connect each device via a plurality of networks or communication lines.

<Structure of target person system>
FIG. 2 is a diagram showing the configuration of the subject system 100. The target person system 100 is an example of a grasping means for grasping the situation of the watching target person. In the configuration example shown in FIG. 2, the subject system 100 includes a camera 101, a microphone 102, a sensor 103, and a communication server 104. The camera 101, the microphone 102, and the sensor 103 are connected to the communication server 104. The communication server 104 is further connected to the center server 200 via the network 400.

The camera 101 is an imaging device that acquires an image of the person to be watched and the space around the person to be watched. The image acquired by the camera 101 is sent to the communication server 104 as digital data. The image acquired by the camera 101 may be either a moving image or a still image, but here, it is assumed that the moving image is acquired. In addition to the moving image data in the moving image format, a set of time-series still image data taken at regular intervals may be acquired. As the camera 101, those having various configurations may be used depending on the usage environment. For example, it may be a portable camera or a fixed camera. Further, a wide-angle camera or an all-around camera that acquires an image of a wide surrounding space, a plurality of camera groups that capture a specific object or place at various angles, or the like may be used. For example, when the person to be watched moves in a plurality of rooms, the camera 101 may be installed in each room. Further, paying attention to the data format of the acquired image, it may be a digital camera that acquires an image as a digital image, or an analog camera that acquires an image as an analog image. In the case of an analog camera, the image acquired as an analog image is converted into digital data and output. The specific configuration of the camera 101 in the subject system 100 is specified according to the environment and scene in which the image is acquired, the restrictions on the devices that can be used, and the like.

The microphone 102 is a recording device that records sound (acoustic information) as information. As the microphone 102, one having various characteristics and configurations may be used depending on the place to be recorded, the type of acoustic information, and the like. For example, when recording room sound or environmental sound, a microphone 102 having low directivity or low directivity is used. Further, when recording the operating sound of a person to be watched over or a specific object, a microphone 102 having a range characteristic matching the range of the voice or the operating sound and having high directivity is used. The acoustic information recorded by the microphone 102 is sent to the communication server 104 as digital data. Further, the microphone 102 itself may have various configurations such as a fixed microphone, a sound collecting microphone, a pin microphone worn by an individual, and a microphone array having strong directivity. The specific configuration of the microphone 102 in the subject system 100 is specified according to the environment and scene in which recording is performed, restrictions on usable devices, and the like.

The sensor 103 is a detection device that detects various information as physical quantities from the person to be watched and the environment around the person to be watched. As the sensor 103, various types may be used depending on the type of information to be detected. For example, a temperature sensor, a humidity sensor, an illuminance sensor, a speed sensor, a vibration sensor and the like are used alone or in combination. Then, by these sensors 103, environmental information such as room temperature and room brightness, biological information such as blood pressure and pulse, equipment such as water and gas, electrical products, furniture, and the usage status of objects used in daily life. , Various information such as information for specifying the position and operating state of the object is acquired. The specific configuration of the sensor 103 in the target person system 100 is specified according to the target for which information is to be acquired and the type of information to be acquired. The information acquired by the sensor 103 is sent to the communication server 104 as digital data.

The communication server 104 is a communication control device that transmits information acquired by the camera 101, the microphone 102, and the sensor 103 to the center server 200. As the communication server 104, for example, a personal computer (PC) or the like is used. Further, using a router that constitutes a LAN to which a camera 101, a microphone 102, and a sensor 103 (hereinafter, collectively referred to as an "information acquisition device") are connected to have an information processing function and a communication control function. Is also good. Further, various information devices (television receiver, facsimile machine, etc.) having a function of connecting to the network 400 and an information processing function may be used.

The communication server 104 collects information from each information acquisition device according to the setting. Specifically, information may be acquired continuously at all times, information may be acquired intermittently at regular time intervals, or when a predetermined event occurs (event). Information may be acquired as a trigger). Information may be sent from the information acquisition device by push-type communication, or information may be acquired by pull-type communication in which the communication server 104 inquires of the information acquisition device. As long as the communication does not collide, the information may be sent to the communication server 104 by a different method or mode for each information acquisition device.

Further, the communication server 104 transmits the acquired information to the center server 200 according to the setting. Specifically, it may be connected to the center server 200 at all times to continuously transmit information, or may be intermittently connected to the center server 200 at regular time intervals to transmit information. Further, information may be transmitted by connecting to the center server 200 as a trigger when a predetermined event occurs. For example, an emergency report may be made when information indicating that an abnormal situation (such as fainting in a room) has occurred to the person to be watched over is obtained. Further, the information may be transmitted to the center server 200 by a different method or mode depending on the type of the device from which the information has been acquired. For example, the moving image acquired by the camera 101 may be continuously transmitted, and the information indicating the usage status of the equipment or the electric product may be collectively transmitted at regular intervals.

Further, the communication server 104 includes a computer as a control means and a processing means and a storage device (memory, magnetic disk device, etc.) as a storage means, stores information acquired from the information acquisition device, and data based on the settings. To process. For example, information acquired from a plurality of sensors 103 may be combined to generate information indicating the behavior of the watching target person (for example, getting up, moving a room, etc.) or information indicating that a specific event has occurred. In addition, data continuously acquired from the information acquisition device may be thinned out, numerical values may be rounded, or simple statistical processing may be performed. The accumulated information may be deleted after being transmitted to the center server 200, or may be stored for a certain period of time or according to the storage capacity of the storage device. The individual information may be erased based on the erasing conditions individually determined according to the type of the device from which the information was acquired.

The camera 101, microphone 102, and sensor 103 shown in FIG. 1 are merely examples of information acquisition devices. In the present embodiment, the camera 101 is indispensable, but other information acquisition devices are appropriately selected according to the environment for collecting information, the type of information to be collected, and the like. Further, the information may be acquired by a device other than the camera 101, the microphone 102 and the sensor 103 shown in FIG. For example, the communication server 104 may store the operation history such as the on / off of the switch of the electric product and the function adjustment as information.

Further, the radio waves used in Wi-Fi or the like may be used as a sensor to detect the movement or movement of people or articles indoors. For example, a correlation can be found between the reflection path of Wi-Fi radio waves and the indoor environment. Therefore, it is possible to detect a change in the indoor environment from a change in the reflection state of radio waves. By using the detection technology using Wi-Fi radio waves in addition to the camera 101, it is possible to detect the movement and movement of people and articles even in areas outside the shooting range of the camera 101, and the camera 101 is used. It is possible to supplement the information obtained in the above. Further, if the detection technology using the Wi-Fi radio wave is used instead of the camera 101, it is possible to perform privacy-friendly detection of a place that is not desired to be visualized.

When the movement or movement of a person or an article is detected by using the detection technology using the Wi-Fi radio wave instead of the camera 101 or the sensor data by the sensor 103, it is detected from the captured image taken by the camera 101. It is conceivable that only movements larger than the movements can be detected. For example, an action in which a person turns his / her face may be extracted from a captured image, but may not be detected by a detection technique using Wi-Fi radio waves. However, paying attention to the usage mode of the monitoring system, it is often sufficient to detect the movement and posture of a person (whether standing, sitting, lying, etc.). Therefore, the detection based on the captured image by the camera 101 may be replaced with the detection based on the detection technology using Wi-Fi radio waves or the like.

<Center server configuration>
The center server 200 is a server that processes an image (captured image) acquired by the camera 101, generates a content image based on the captured image, and provides the watcher terminal 300 with the content image. Specifically, the center server 200 generates a background image representing a virtual space imitating an area (real space) captured in the captured image and a character image set in association with the person captured in the captured image. Then, these are combined to generate a content image. As a result, in the watcher terminal 300, the captured image is not displayed as it is, but is replaced with a content image in which the virtual space and the character are combined and displayed.

FIG. 3 is a diagram showing a functional configuration of the center server 200. The center server 200 includes an information acquisition unit 201 as acquisition means, and an image analysis unit 202, a motion recognition unit 203, an acoustic analysis unit 204, a voice analysis unit 205, and a sensor / data analysis unit 206 as analysis means. Be prepared. Further, the center server 200 includes a background information database (DB) 207, a virtual space construction unit 208, and a virtual space management unit 209 as virtual space management means. Further, the center server 200 includes a character information database (DB) 210, a character setting unit 211, and a character control unit 212 as character management means. Further, the center server 200 includes a content generation unit 213 and an output unit 214 as content output means. The center server 200 is not limited to a single server machine, and may be a system in which the above functions are distributed among a plurality of servers.

The information acquisition unit 201 is a communication means for acquiring information from the communication server 104 of the target person system 100 via the network 400. As a communication mode, the information may be continuously received by connecting to the communication server 104 at all times, or may be intermittently connected to the communication server 104 to receive the information at regular time intervals. Further, as a communication method, information may be sent from the communication server 104 by push-type communication, or information may be acquired by pull-type communication in which the center server 200 inquires to the communication server 104. The information acquisition unit 201 is an example of an acquisition means for acquiring information of a person to be watched over.

The image analysis unit 202 is a processing means for analyzing the captured image captured by the camera 101 among the information acquired from the subject system 100. The image analysis unit 202 analyzes the captured image and recognizes the captured region as a three-dimensional space. In addition, the image analysis unit 202 identifies a person and a non-person object among the photographed objects. In addition, the image analysis unit 202 identifies at least a person to be watched over from among the photographed persons. Further, the image analysis unit 202 may identify an individual person other than the person to be watched over. Further, the image analysis unit 202 may identify an animal among objects other than a person. Further, furniture, fittings, other articles and the like may be identified. Identification of a person or other object by image analysis can be performed using existing technology related to image recognition. Regarding the identification of the person to be watched over or a specific person, in addition to or instead of the image analysis by the image analysis unit 202, an IC (integrated circuit) card on which personal identification information (ID) is recorded is possessed. Identification may be performed based on the personal identification information acquired from this IC card.

The motion recognition unit 203 is a processing means for recognizing the motion of a person or the like identified by the image analysis unit 202. The motion recognition unit 203 recognizes the motion of the target identified as a person (at least the person to be watched over) based on the analysis result of the captured image by the image analysis unit 202. Specifically, the motion recognition unit 203 makes a person partial image based on, for example, a shape change of a target portion (hereinafter, referred to as “person partial image”) identified as a person in the captured image with the passage of time. Recognize what the corresponding person is doing. For motion recognition, various existing techniques such as pattern recognition and motion identification based on a skeleton model may be used. Further, when the image analysis unit 202 identifies an animal, the motion recognition unit 203 may recognize the motion of the identified animal.

The acoustic analysis unit 204 is a processing means for analyzing the acoustic information recorded by the microphone 102 among the information acquired from the subject system 100. The acoustic analysis unit 204 analyzes the recorded acoustic information and identifies a human voice. Further, the acoustic analysis unit 204 may identify a specific sound or animal bark that is predetermined as environmental information from the acoustic information.

The voice analysis unit 205 is a processing means for analyzing the voice of a person identified by the acoustic analysis unit 204. The voice analysis unit 205 analyzes the voice and recognizes the utterance content. The voice analysis unit 205 may extract voice commands based on the recognition result of the utterance content. A voice command is a command issued by voice. For example, there is a voice saying "Turn on the light" to turn on the lighting switch, a voice saying "Turn down the room temperature" to adjust the set temperature of the air conditioner, and the like. Further, the voice analysis unit 205 may analyze the voice to identify the speaker.

The sensor data analysis unit 206 is a processing means for analyzing the data (sensor data) acquired by the sensor 103 among the information acquired from the subject system 100. The sensor data analysis unit 206 performs preset data processing according to the type of sensor data (or the type of the sensor 103 from which the sensor data has been acquired) and extracts information. The extracted information is provided to the watcher terminal 300. In addition, the extracted information is taken into consideration as necessary in analysis by other analysis means and character control by the character control unit 212.

The background information DB 207 is a database that stores component data used for constructing a virtual space. The background information DB 207 stores various component data for constructing a virtual space imitating a real space. Specifically, for example, parts that represent areas (places) such as the ground, floor, walls, and ceiling, parts that represent articles placed in areas such as furniture and electrical appliances, and equipment installed in a room are represented. Data such as parts to be used, parts to represent animals that are not represented as plants or characters, etc. are stored. The component data is three-dimensional computer graphics data (3DCG data). These parts may be prepared assuming an indoor or outdoor space set according to various themes. For example, as an indoor space, parts expressing items that are expected to exist in such a space may be prepared under the theme of Western-style room, Japanese-style room, living room, bedroom, kitchen, corridor, entrance, and the like. .. As the outdoor space, parts expressing articles and the like that are expected to exist in such a space may be prepared under the theme of parks, plazas, fields, forests, underwater, outer space, and the like. Further, even if the theme is the same, a plurality of parts groups having different realism, degree of exaggeration, degree of simplification, and the like may be prepared. For example, for the same theme, a more realistically constructed component group and a more stylized component group may be prepared.

The virtual space construction unit 208 is a processing means for constructing a virtual space using the component data held in the background information DB 207. The virtual space construction unit 208 constructs a virtual space imitating the real space which is the target area of the watching service. The target area of the watching service is an area set as a place where the watching target person should be watched. Specifically, for example, an area where the person to be watched mainly stays or acts, a route where the person to be watched frequently moves, an area to which the person to be watched moves, and the like can be mentioned. A plurality of target areas for the monitoring service may be set for one monitoring target person. In this case, a number of virtual spaces are constructed according to the number of target areas of the watching service. Information on the real space, which is the target area of the monitoring service, can be obtained from, for example, the analysis result of the image analysis unit 202. The data in the virtual space is generated as 3DCG data in the three-dimensional space.

In the virtual space constructed by the virtual space construction unit 208, parts corresponding to articles, equipment, and the like existing in the prototype real space (hereinafter, referred to as “corresponding parts”) are arranged. Here, the virtual space may be constructed according to the theme described in the above description of the background information DB 207. In this case, the corresponding parts arranged in the virtual space are in line with the theme set in the virtual space. For example, when a chair exists in a real space and a virtual space with a forest theme is constructed, the chair may be replaced with a stump (part) and arranged. As a result, the virtual space is constructed as a space that reflects the state of the target area of the watching service.

It should be noted that the state of the original real space and the state of the virtual space do not have to exactly match. For example, a virtual space with the theme of an outdoor space such as a forest or a field may be constructed with respect to a real space that is indoors (for example, a living room). Further, the article or the like in the real space and the parts arranged in the virtual space do not have to be the same, and the chair may be replaced with a stump as described above.

The virtual space management unit 209 is a management means for managing the virtual space constructed by the virtual space construction unit 208. The virtual space management unit 209 manages the virtual space constructed by the virtual space construction unit 208 as a background space for replacement with respect to the target area of the monitoring service. Specifically, for example, the 3DCG data of the constructed virtual space is stored in a storage device (not shown), and is managed in association with the identification information that identifies the real space of the target area of the monitoring service, which is the prototype of the virtual space. That is, the virtual space management unit 209 is an example of the background image holding means. The virtual space management unit 209 accepts updates due to the reconstruction of the virtual space construction unit 208 with respect to the managed virtual space. Further, when a plurality of virtual spaces corresponding to one of the target areas of the watching service are constructed, the virtual space management unit 209 receives an instruction from an external device (for example, the watching person terminal 300) and replaces the virtual space. Switch the virtual space used as the background space. That is, the virtual space management unit 209 is an example of a background selection receiving means that accepts the selection of a background image that replaces the background of the person to be watched over. In addition, the virtual space management unit 209 accepts editing and updating of the virtual space manually by the user through an appropriate user interface.

Editing of the virtual space by the user's manual operation is performed, for example, as follows. First, as a user interface, a terminal device (for example, a watcher terminal 300) in which graphic software capable of displaying 3DCG in a virtual space is installed is prepared. This terminal device is connected to the center server 200, and the virtual space management unit 209 is instructed to call the 3DCG data of the virtual space to be edited. Then, the terminal device displays the called 3DCG, accepts the user's operation through the interface of the graphic software, and edits the 3DCG. As a result, it is possible to change the arrangement of the parts arranged in the virtual space or replace the parts with other parts by manual operation. Further, in the virtual space, some parts may be arranged in a place where the article does not exist in the real space, or conversely, the part may be arranged as a space without arranging the parts in the place where the article exists in the real space. The content edited by the manual operation is also reflected when the virtual space is updated by the reconstruction of the virtual space construction unit 208.

The character information DB 210 is a database that stores character data operating in the virtual space managed by the virtual space management unit 209. The character information DB 210 stores an overall image of various characters and data for each body part. The character data is 3DCG data. Further, a character based on two-dimensional graphics data may be added. As the character, a plurality of characters having different designs for the same type of person may be prepared. Further, a character according to the theme corresponding to the theme of the component data stored in the background information DB 207 may be prepared. For example, it may be a character whose motif is a robot, anthropomorphic animal, a doll, or the like. Further, various characters having different realism, degree of exaggeration, degree of simplification, and the like may be prepared.

The character setting unit 211 is a processing means for setting a character operating in the virtual space managed by the virtual space management unit 209 by using the character data held in the character information DB 210. The character setting unit 211 sets a character that replaces the watching target person and other persons in the image taken by the camera 101 of the target person system 100. Further, when an animal is identified by the image analysis unit 202, the character setting unit 211 may set a character that replaces the identified animal. The character is set by combining the overall image of the character and the data of each part of the body held in the character information DB 210. The character to be set is selected according to, for example, the theme of the virtual space managed by the virtual space management unit 209. Further, a character selection instruction to be set may be received from an external device (for example, the watcher terminal 300). The character information DB 210 and the character setting unit 211 are examples of character image holding means. Further, the character setting unit 211 is an example of the character selection receiving means.

The character control unit 212 is a processing means for controlling the operation of the character set by the character setting unit 211. The character control unit 212 controls the character and changes the state based on the situation of the person recognized by the motion recognition unit 203 from the image taken by the camera 101 of the target person system 100. Specifically, the corresponding character is moved or moved according to the movement or movement of the person. The analysis result by the voice analysis unit 205 and the analysis result by the sensor data analysis unit 206 are also taken into consideration in the control of the character, if necessary. For example, when it is detected from the analysis result of acoustic information that a specific person has spoken, control may be performed to move the mouth of the character corresponding to this person in accordance with the timing of the utterance. Further, when it is detected from the sensor data that the body temperature of the watching target person is high, control may be performed to display the face of the character corresponding to the watching target person in red. Further, the character control unit 212 controls the operation of the character in the virtual space selected as the background space corresponding to the target area of the watching service. Therefore, the movement of the character is restricted by the size and shape of the virtual space selected as the background space, the arranged parts, and the like. For example, if a part is placed on the path on which the character moves, the character is moved so as to avoid the part.

Further, the character control unit 212 may perform a predetermined effect on the state of the character within a range that does not deviate from the situation of the person corresponding to the character. For example, when a person is walking in a room, the character corresponding to the person may be moved by an action like the selected character instead of walking like a human. Also, if the person is lying on the bedding and the sensor data obtained from the biosensor shows a resting state, the character is not just laid down, but as if it were sleeping. You may control it. In addition, for situations that cannot be acquired by the information acquisition device of the target person system 100 or situations that are not important from the viewpoint of watching, it is possible to perform an effect according to the character corresponding to the person. For example, a person standing or sitting in a chair is reflected in the corresponding character's movements, while face orientation and arm movements are not based on the person's movements, but on the selected character. You may replace it with a behavior that seems to be.

The content generation unit 213 is a processing means for generating a content image by synthesizing a virtual space selected as a background space corresponding to the target area of the watching service and a character set by the character setting unit 211. The content generation unit 213 generates, as a content image, a moving image showing how the character operates under the control of the character control unit 212 in the virtual space selected as the background space. That is, the content generation unit 213 is an example of the image generation means. The content image is generated as, for example, a 3DCG animation. Acoustic data such as audio and environmental sound extracted by the acoustic analysis unit 204 and the audio analysis unit 205 are combined with the content image. At this time, the extracted acoustic data such as voice may be synthesized as it is, or may be synthesized by performing a process such as frequency conversion. Further, based on the analysis results of the acoustic analysis unit 204 and the voice analysis unit 205, a specific spoken voice or generated sound may be erased and synthesized. Further, a specific spoken voice or sound effect may be generated and combined with the content image according to the movement of the character controlled by the character control unit 212.

The output unit 214 is a processing means for outputting the content image generated by the content generation unit 213 to the watcher terminal 300. The output unit 214 connects to the watcher terminal 300 via the network 400, and transmits, for example, a content image by 3DCG animation by a streaming method.

<Configuration of watcher terminal>
The watcher terminal 300 is a presentation means for presenting the information of the watch target person acquired by the target person system 100 and the center server 200 to the watcher. The watcher terminal 300 has an output means for outputting the watcher's information by image or sound.

FIG. 4 is a diagram showing a functional configuration of the watcher terminal 300. The watcher terminal 300 includes a content acquisition unit 301, a control unit 302, and a display unit 303 and an acoustic output unit 304 as output means. Although not particularly shown, the watcher terminal 300 may include a storage device that holds the information of the watch target person acquired from the center server 200. The watcher terminal 300 is realized by various information processing devices such as a personal computer (PC) and a tablet-type information terminal, for example.

The content acquisition unit 301 is a processing means for acquiring a content image related to the person to be watched from the center server 200. The content acquisition unit 301 connects to the center server 200 via the network 400, and acquires, for example, a content image which is a 3DCG animation transmitted by a streaming method. When the watcher terminal 300 has a storage device, the acquired content image may be held in the storage device.

The control unit 302 is a control means for performing display control of the content image acquired by the content acquisition unit 301 and output (reproduction) control of acoustic information. For example, the control unit 302 adjusts the image size according to the size of the display area of the content image in the display unit 303, and displays the adjusted content image in the display area of the display unit 303. Further, when the content image is held in the storage device, the content image may be read out from the storage device and displayed on the display unit 303.

The display unit 303 is an output means for displaying the content image. That is, the display unit 303 is an example of the display means. The display unit 303 is a display device such as a liquid crystal display. The display unit 303 may display the content image with the entire display screen as the display area, or may display the content image in a display area such as a window set as a part of the display screen. Further, when the content image is held in the storage device, the display unit 303 may display the content image read from the storage device under the control of the control unit 302.

The acoustic output unit 304 is an output means for performing acoustic output based on the acoustic data synthesized in the content image. Further, when the content image is held in the storage device, the acoustic output unit 304 may perform acoustic output based on the acoustic data of the content image read from the storage device under the control of the control unit 302. The sound output unit 304 may emit a warning sound when an abnormal situation occurs in the person to be watched over.

<Display example of content image>
5A and 5B are diagrams showing a display example of a content image, FIG. 5A is a diagram showing a state in which a captured image of a person to be watched is displayed as it is on the display unit 303, and FIG. 5B is a diagram showing a state. FIG. 5 is a diagram showing a state in which a content image generated based on the captured image of FIG. 5A is displayed on the display unit 303. In the photographed image shown in FIG. 5A, it can be seen that there are two persons, one elderly man and one woman. Further, as a background, it can be seen that the room has a sofa arranged in an L shape, a TV receiver (display device), and a stand on which the TV receiver (display device) is placed. And you can see that two people are sitting on the sofa. On the other hand, in the content image shown in FIG. 5B, it can be seen that there are two people as people. Further, as a background, it can be seen that the room has a chair, a TV receiver (display device), and a stand on which the TV receiver (display device) is placed. And it can be seen that two people are sitting in the chair.

As described above, the content image is an image generated by stylizing a display target (a person, an article in the background, etc.) based on the captured image. For this reason, the displayed content is abstracted and some information is lost. In the watching service, the person to be watched may have a psychological resistance based on an association that allows them to peep into their own lives. In addition, it is a great psychological burden for the watcher to continue to view the captured image having a large amount of information including the private content of the watchee. In the present embodiment, by using the content image as shown in FIG. 5B as the image displayed on the watcher terminal 300, the psychological resistance of the watcher is reduced and the psychological burden of the watcher is burdened. Is also expected to be reduced.

Further, as described above, in the present embodiment, the content image generated by the center server 200 can be composed of a combination of a background image and characters selected according to a specific theme. As a result, the content image becomes a more abstract and decorative image, which further reduces the psychological resistance of the person being watched over, and realizes a watching service with an easy-to-see image that the watcher can feel warmth. can do.

FIG. 6 is a diagram showing an example of a content image composed of a specific theme. The content image shown in FIG. 6 is an example of an image with the theme of fairy tale expression, in which the background is a field and the person is replaced with a cartoon-designed animal character. In the content image of FIG. 6, a bear character and a rabbit character are each sitting on a stump in the background image of the field. From this content image, it is recognized that there are two people in the range (the target area of the watching service) taken by the camera 101 of the target person system 100, and each of them seems to be sitting on a chair.

FIG. 7 is a diagram showing an example of a display mode of the content image. FIG. 7 shows an example of a display mode when a personal computer (PC) is used as the watcher terminal 300 and a PC display device is used as the display unit 303. On the display screen of the display unit 303 shown in FIG. 7, a desktop environment as a work area by a PC is configured, and windows 303a and 303b as display areas are displayed. An electronic document is displayed in the window 303a, and a content image received from the center server 200 is displayed in the window 303b. By displaying the content image on a part of the display screen as in the display example shown in FIG. 7, the watcher does not always look at the content image, but performs other work on the screen of the display unit 303. You can watch over while watching.

In the present embodiment, since the stylized content image is used as the image to be displayed on the watcher terminal 300, the trivial information included in the captured image is reduced. As a result, it is expected that even if the image is displayed together with other working images as in the display example shown in FIG. 7, it will be easy for the watcher to see. Further, in the content image, since the content of the image is abstracted as compared with the captured image, the situation of the person to be watched over even when the content image is displayed in a small display area as shown in the window 303b shown in FIG. Easy to grasp. In addition, based on the information obtained from the target person system 100, when a situation that requires an urgent response to the watching target person occurs, switching control is performed so that the content image is displayed on the entire display screen of the display unit 303. Or, by displaying a warning message, the attention of the watcher may be drawn.

<Other Embodiments>
The watching system according to the above embodiment has a configuration in which information is transmitted in one direction from the target system 100 to the watching terminal 300 only. On the other hand, a configuration that realizes bidirectional information transmission that sends information from the watcher terminal 300 to the target person system 100 may be used. In this case, for example, a means for acquiring information is provided in the watcher terminal 300, and a means for outputting information is provided in the target person system 100. Then, the information acquired by the watcher terminal 300 is transmitted to the output means of the target person system via the center server 200.

FIG. 8 is a diagram showing a configuration of a watcher terminal 300 when bidirectional information transmission is performed. The watcher terminal 300 shown in FIG. 8 includes a content acquisition unit 301, a display unit 303, an acoustic output unit 304, a control unit 310, an imaging unit 311, an acoustic acquisition unit 312, and a transmission unit 313. In the configuration example shown in FIG. 8, the content acquisition unit 301, the display unit 303, and the acoustic output unit 304 are the same as the respective configurations described with reference to FIG. 4, and therefore the same reference numerals are given and the description thereof will be omitted.

The control unit 310 controls the display of the content image acquired by the content acquisition unit 301 and the output (reproduction) control of the acoustic information, and also controls the image acquired by the imaging unit 311 and the acoustic information acquired by the sound acquisition unit 312. It is a control means for performing transmission control. The functions of the content image display control and the acoustic information output control in the control unit 310 are the same as those of the control unit 302 described with reference to FIG.

The control unit 310 transmits the image acquired by the image pickup unit 311 and the acoustic information acquired by the sound acquisition unit 312 to the center server 200 via the transmission unit 313. Specifically, it may be connected to the center server 200 at all times to continuously transmit information, or may be intermittently connected to the center server 200 at regular time intervals to transmit information. Further, information may be transmitted by connecting to the center server 200 as a trigger when a predetermined event occurs.

The image pickup unit 311 is an image pickup means for acquiring an image of the user (watcher) of the watcher terminal 300 and the space around the user (watcher). The image acquired by the image pickup unit 311 may be either a moving image or a still image, but here, it is assumed that the moving image is acquired. In addition to the moving image data in the moving image format, a set of time-series still image data taken at regular intervals may be acquired. As the image pickup unit 311, for example, a built-in camera provided in many notebook PCs, tablet information terminals, and the like may be used, or the camera (external camera) as an external device may be used as the watcher terminal 300. It may be connected and used. When an external camera is used, the image taken by the external camera is sent as digital data to the watcher terminal 300 (control unit 310). When an external camera is used, cameras having various configurations can be used, such as the camera 101 used in the subject system 100.

The sound acquisition unit 312 is a recording device that records sound (acoustic information) as information. As the sound acquisition unit 312, for example, a built-in microphone provided in many notebook PCs, tablet-type information terminals, and the like may be used, or a microphone (external microphone) as an external device may be used as a watcher terminal 300. It may be used by connecting to. When using an external microphone, microphones having various configurations can be used, such as the microphone 102 used in the subject system 100.

The transmission unit 313 is a transmission means for transmitting the image acquired by the image pickup unit 311 and the acoustic information acquired by the sound acquisition unit 312 to the center server 200 under the control of the control unit 310. The transmission unit 313 is realized by a network interface connected to the network 400.

When the center server 200 receives the image and acoustic information from the watcher terminal 300, it analyzes and processes the image and acoustic information in the same manner as the image and acoustic information acquired from the subject system 100, and generates a content image by 3DCG animation. Then, the center server 200 transmits the generated content image to the target person system 100 by a streaming method.

FIG. 9 is a diagram showing a configuration of a target person system 100 when bidirectional information transmission is performed. The subject system 100 shown in FIG. 9 includes a camera 101, a microphone 102, a sensor 103, a communication server 110, a display device 111, and an audio device 112. In the configuration example shown in FIG. 9, the camera 101, the microphone 102, and the sensor 103 are the same as the respective configurations described with reference to FIG. 2, and therefore the same reference numerals are given and the description thereof will be omitted.

The communication server 110 is a communication control device that transmits the information acquired by the camera 101, the microphone 102, and the sensor 103 to the center server 200, and also sends the information received from the center server 200 to the display device 111 and the sound device 112. The function of the communication server 110 to acquire information from each information acquisition device and transmit it to the center server 200 is the same as that of the communication server 104 described with reference to FIG.

The communication server 110 acquires image data from the center server 200 and sends it to the display device 111, and acquires acoustic data from the center server 200 and sends it to the acoustic device 112. Specifically, the communication server 110 connects to the center server 200 via the network 400, and acquires, for example, a content image which is a 3DCG animation transmitted by a streaming method. Then, the communication server 110 separates the received content image data into image data and acoustic data, and sends the data to the display device 111 and the acoustic device 112.

The display device 111 is an output means for displaying the content image. The display device 111 is realized by, for example, a liquid crystal display or the like. The display device 111 may display the content image with the entire display screen as the display area, or may display the content image in a display area such as a window set as a part of the display screen. Further, the content image may be held in a storage device (not shown) such as a magnetic disk device, and the content image may be read out from the storage device and displayed.

The acoustic device 112 is an output means for performing acoustic output based on acoustic data synthesized in a content image. The audio device 112 is realized by, for example, a speaker connected to the communication server 110. The content image may be held in a storage device (not shown) such as a magnetic disk device, the acoustic data of the content image may be read from the storage device, and acoustic output may be performed based on the acoustic data of the read content image. The sound device 112 may emit a warning sound when an abnormal situation occurs in the person to be watched over.

FIG. 10 is a diagram showing a display example of content images of the watcher terminal 300 and the target person system 100 in the case of bidirectional information transmission. Here, it is assumed that an elderly couple is to be watched over and a watching system is installed as a watching over the child. That is, it is assumed that the target person system 100 is installed in the residence of the elderly couple and the watcher terminal 300 is installed in the residence of the child. As shown in FIG. 10, the display unit 303 of the watcher terminal 300 displays a content image based on a photographed image (an image of an elderly couple photographed) taken by the camera 101 of the subject system 100. There is. On the other hand, on the display device 111 of the subject system 100, a photographed image (a woman who is a child of an elderly couple and their children (grandchildren of an elderly couple)) photographed by the imaging unit 311 of the watcher terminal 300 is captured. ) Is displayed.

In this way, by transmitting information in both directions and confirming each other's situation, it is expected that the person being watched over will be able to recognize the person being watched over, and that the feeling of psychological resistance will be further reduced. .. Also, for the person to be watched over, the obligatory character of unilaterally watching over the person to be watched over has diminished, and it is possible to watch over the other person while having a two-way dialogue, further reducing the psychological burden. Can be expected.

In the other embodiment, the image and acoustic information acquired by the watcher terminal 300 are processed by the center server 200 and provided to the target person system 100 as a content image. Therefore, the center server 200 can manage and process both the information acquired by the target person system 100 and the information acquired by the watcher terminal 300. Therefore, control may be performed so that the information acquired from one of them affects the content image based on the other information. For example, the watching target person speaks to the character displayed on the display device 111, and the center server 200 analyzes the information acquired from the target person system 100 to extract this action of the watching target person. Then, the center server 200 causes the character corresponding to the watcher to respond by waving in the content image based on the information acquired from the watcher terminal 300, regardless of the behavior of the watcher. By performing such control, even if the watcher does not respond every time, the character corresponding to the watcher reacts to the action of the watcher, so that the watcher's interest is enhanced and the psychology of the watch system is enhanced. It is expected that the feeling of resistance will be further reduced. In addition, although an example in which the character corresponding to the watcher reacts to the action of the watcher is described here, on the contrary, the character corresponding to the watcher reacts to the action of the watcher. You may.

<Modification example>
In the other embodiment described above, the watcher terminal 300 is provided with information acquisition means, the target person system 100 is provided with information output means, and the target person system 100 outputs the content image based on the information acquired by the watcher terminal 300. It was configured. On the other hand, a system configuration that realizes mutual information transmission can be considered by providing the target person system 100 on the watcher side and the watcher terminal 300 on the watcher side as well. In other words, it is a system that watches over each other as a watcher and a watcher. Also in this case, by relaying the bidirectional information transmission on the center server 200, it is possible to control the information acquired from one of the above-mentioned information to affect the content image based on the information of the other. In addition, if the system is configured to watch over each other, it is possible for three or more target persons to watch over each other. As an example, a target person system 100 and a watcher terminal 300 are provided at each target person's place of residence, and a content image based on information acquired from the target person system 100 of each other target person is output by the own watcher terminal 300. It may be configured to be used.

Further, in each of the above embodiments, the information acquired by the information acquisition means in the subject system 100 is analyzed by the center server 200, and a content image is generated based on the analysis result. On the other hand, in the target person system 100, for example, the communication server 104 may be provided with an information analysis function, the information acquired by the information acquisition means may be analyzed, and the analysis result may be transmitted to the center server 200. For example, the image captured by the camera 101 is subjected to the same processing as the image analysis unit 202 and the motion recognition unit 203 of the center server 200 described with reference to FIG. Then, the information of the recognized operation is transmitted from the communication server 104 to the center server 200. The acoustic information recorded by the microphone 102 is also processed in the same manner as the acoustic analysis unit 204 and the voice analysis unit 205 of the center server 200, and the obtained analysis result is transmitted from the communication server 104 to the center server 200. Similarly, the sensor data obtained by the sensor 103 is subjected to the same processing as the sensor data analysis unit 206 of the center server 200, and the obtained processing result is transmitted from the communication server 104 to the center server 200. In this way, only the information necessary for generating the content image is transmitted from the target person system 100 to the center server 200. Therefore, the communication traffic between the target person system 100 and the center server 200 can be reduced. Further, as compared with the case where the information acquired by the information acquisition means is sent to the center server 200 as it is, the information unnecessary for generating the content image is omitted, so that a monitoring system that considers the privacy of the person to be monitored is realized. be able to.

The content of the analysis process performed by the target person system 100 is not limited to the above process, and may be determined according to the processing capacity and settings of the device that performs the analysis process. For example, even if the subject system 100 performs only the analysis corresponding to the function of the image analysis unit 202 on the image captured by the camera 101, and the motion recognition unit 203 of the center server 200 recognizes the motion of a person or the like. good. Similarly, for the acoustic information recorded by the microphone 102, only the analysis corresponding to the function of the acoustic analysis unit 204 may be performed by the subject system 100, and the voice analysis may be performed by the voice analysis unit 205 of the center server 200. .. Further, with respect to the sensor data, a part of the analysis may be performed by the subject system 100, and the other analysis may be performed by the center server 200.

The analysis process in the subject system 100 may be performed by the communication server 104 provided with the information processing function as described above, or may be performed by providing the information processing function in a device used as an information processing means such as a camera 101. Further, each device of the information acquisition means and the communication server 104 may perform the analysis processing in a distributed manner (a part of the analysis is performed by each device, and the other analysis is performed by the communication server 104, etc.).

Further, in the watcher terminal 300, the functions of the background information DB 207, the virtual space construction unit 208, the virtual space management unit 209 character information DB 210, the character setting unit 211, and the character control unit 212 of the center server 200 described with reference to FIG. It may be configured to include. Then, by analyzing the information acquired by the target person system 100 and combining it with a configuration in which only the information necessary for generating the content image is transmitted, the target person system 100 does not involve the center server 200. It is possible to realize a watching system that directly transmits information from the watching person terminal 300 to the watching person terminal 300.

Although the embodiments of the present invention have been described above, the technical scope of the present invention is not limited to the above embodiments. For example, in the above embodiment, it is possible to select a virtual space or a character according to various themes with respect to the background space or the character. Here, a background space and character selection means are provided in the watcher terminal 300, and based on the selection instruction from the watcher terminal 300, the background space is displayed at any time even while the content image is being displayed on the center server 200. The virtual space and the character used as may be switched. Further, the content image may be selected in units of themes, and the background space and characters of the theme desired by the watcher may be freely switched. Further, the target person system 100 may also be configured to provide a background space and character selection means so that the watching target person can select the background space and the character to be displayed in the content image. In addition, various changes and alternative configurations that do not deviate from the scope of the technical idea of the present invention are included in the present invention.

100 ... Target system, 101 ... Camera, 102 ... Microphone, 103 ... Sensor, 104 ... Communication server, 200 ... Center server, 201 ... Information acquisition unit, 202 ... Image analysis unit, 203 ... Motion recognition unit, 204 ... Acoustic analysis Department, 205 ... Voice analysis unit, 206 ... Sensor data analysis unit, 207 ... Background information database (DB), 208 ... Virtual space construction unit, 209 ... Virtual space management unit, 210 ... Character information database (DB), 211 ... Character setting unit, 212 ... Character control unit, 213 ... Content generation unit, 214 ... Output unit, 300 ... Watcher terminal, 301 ... Content acquisition unit, 302 ... Control unit, 303 ... Display unit, 304 ... Acoustic output unit

Claims (6)

A means of grasping the situation of the person to be watched over,
A background image holding means for holding a background image that replaces the background of the person to be watched over,
A character image holding means for holding a character image that replaces the person to be watched over,
A display means for expressing the situation of the person to be watched as grasped by the grasping means by a character image arranged on the background image, and a display means.
Equipped with a,
The situation include situations of the watching target's background in the real space, the replacement of the background image of the background, the Rukoto contents and location of goods and equipment that are present in the real space are considered A featured watching system. The monitoring system according to claim 1, wherein the display means changes the displayed state of the character according to the situation of the monitoring target person grasped by the grasping means. The display means is characterized in that, in addition to the change in the state of the character according to the situation of the watching target person grasped by the grasping means , a predetermined effect unique to the character is displayed for each character. The monitoring system according to claim 2. Further provided with a background selection accepting means for accepting the selection of a background image to replace the background of the person to be watched over.
The display means is characterized in that, among the background images held by the background image holding means, the background image set according to the selection result received by the background selection receiving means is displayed. The monitoring system according to claim 3. Further provided with a character selection receiving means for receiving the selection of a character image to replace the watching target person,
The display means is characterized in that, among the character images held by the character image holding means, the character image set according to the selection result received by the character selection receiving means is displayed. The monitoring system according to claim 4. An acquisition method for acquiring information on the person to be watched over,
A background image holding means for holding a background image that replaces the background of the person to be watched over,
A character image holding means for holding a character image that replaces the person to be watched over,
An image generation means for generating an image showing the situation of the watching target person by combining the background image and the character image based on the information of the watching target person acquired by the acquisition means.
Equipped with a,
The situation include situations of the watching target's background in the real space, the replacement of the background image of the background, the Rukoto contents and location of goods and equipment that are present in the real space are considered A featured watching system.

Images