JP6460762B2 - Home appliance network system - Google Patents

Description

  The present invention relates to a home appliance network system in which a sensor having a communication function and a home appliance (household electronic device) are connected via a communication network.

  Technology to connect communication terminal devices such as mobile phones and smartphones and home appliances via a communication network, remotely control home appliances with communication terminal devices, and remotely check the operating status of home appliances with communication terminal devices (For example, Patent Document 1).

  The communication terminal device is equipped with a camera and sensors for acquiring position information. Therefore, the communication terminal device can also be expressed as a sensor with a communication function that can be carried (hereinafter referred to as a sensor with a portable communication function).

  Home appliances that speak based on information sensed using a built-in sensor (hereinafter referred to as sensing information) have also appeared on the market. Sensing is measurement and discrimination using a sensor.

  In addition, when the communication terminal device and the refrigerator are connected via a communication network and the user requests an image in the store, the image in the store (sensing information) captured by the camera (sensor) installed in the store is sent to the user. Services such as displaying information on stores that provide food at a low price when a user writes ingredients that are transmitted and that the user thinks are insufficient are proposed (for example, patents) Reference 2).

  Furthermore, an indoor environment detection device is provided in the room, and the operation of the air conditioner installed in the room is controlled based on indoor sensing information from the device and a power control signal transmitted from an external power supply destination. There is also a control device for an air conditioner (Patent Document 3).

Japanese Patent Laying-Open No. 2006-140691 (released on June 1, 2006) JP 2002-243335 A (published on August 28, 2002) Japanese Patent Laid-Open No. 10-030834 (published February 3, 1998)

  However, in the above-described conventional technology, sensors that can use sensing information by home appliances are limited to sensors built in the home appliances or dedicated sensors installed indoors to control the operation of the home appliances.

  Therefore, in order for home appliances to use other sensing information, it is necessary to additionally provide another sensor, which is costly. Further, in order to be able to add another sensor, it is necessary to prepare an expansion port or the like at the design stage, which also increases costs. Furthermore, since the place where another sensor is installed is limited to the inside of the home appliance or the installation space of the home appliance, information sensed in an environment away from the home appliance cannot be used.

  The present invention has been made in view of such problems. The purpose of the present invention is to increase the sensing information that can be used by home appliances without adding dedicated sensors inside the home appliance or in the installation space thereof. It is to provide a home appliance network system that can be improved.

  A home appliance network system according to an aspect of the present invention is a home appliance network system in which home appliances, a server device, and a communication terminal device are connected via a communication network, and the communication terminal device includes sensing information of a sensor included in the home appliance network system. Is provided with an information posting unit that receives and executes a process of transmitting the information to the server device at a desired timing, and the server device uses information for determining an operation content commanded to the home appliance from the sensing information received from the communication terminal device. An extraction unit for extracting; the server device or home appliance includes a determination unit that determines an operation content of the home appliance based on information extracted by the extraction unit; and the home appliance is determined by the determination unit The operation content is executed.

  According to the home appliance network system according to one aspect of the present invention, it is possible to improve the convenience of home appliances by increasing the sensing information that can be used by home appliances without adding dedicated sensors inside the home appliance or in the installation space thereof. Play.

It is a figure which shows schematic structure of the household appliance network system concerning Embodiment 1 of this invention. It is a figure which shows schematic structure of the cloud server in the said household appliance network system. It is a figure which shows schematic structure of the communication terminal device in the said household appliance network system. It is a figure which shows an example of the household appliance selection screen for selecting the household appliance which posts the sensing information displayed on the said communication terminal device. It is a figure which shows an example of the information selection screen for selecting the sensing information contributed to the household appliance displayed on the said communication terminal device. It is a figure which shows schematic structure of the household appliance in the said household appliance network system. It is a figure which shows an example of a process of each apparatus contained in the said household appliance network system. It is a figure which shows an example of a process of each apparatus contained in the household appliance network system concerning Embodiment 5 of this invention. It is a figure which shows an example of a process of each apparatus contained in the household appliance network system concerning Embodiment 6 of this invention. It is a figure which shows schematic structure of the communication terminal device in the household appliance network system concerning Embodiment 7 of this invention. It is a figure which shows an example of the pop-up screen which can request | require proposal of the special cooking menu displayed on the communication terminal device in the household appliance network system of the said Embodiment 7. FIG.

[Embodiment 1]
An embodiment of the present invention will be described with reference to FIGS. In this embodiment, a configuration in which a home appliance network system is incorporated in a remote operation system that remotely controls home appliances using a communication terminal device is illustrated, but a combination with the remote operation system is not essential. Moreover, in this Embodiment, the picked-up image (sensing information) by the camera (sensor) with which a communication terminal device is provided is posted to an air conditioner (air conditioner), a microwave oven, a refrigerator, a washing machine, and an air purifier as home appliances. The configuration to be used is illustrated.

(Configuration of home appliance network system 100)
FIG. 1 is a diagram showing a schematic configuration of a home appliance network system 100 according to the present embodiment. As shown in FIG. 1, in the home appliance network system 100, home appliances (electronic devices) 10 (10-1, 10-2, 10-3, 10-4, 10-5, 0- 6), the cloud server (server device) 20, and the communication terminal device 30 (30-1, 30-2, 30-3) are connected via the wide area communication network 62.

  The home appliance 10 is a network home appliance that is connected to the wide-area communication network 62 and can be controlled via the wide-area communication network 62. The home appliance 10 may be configured as a network home appliance by connecting a communication adapter having a communication function to a home appliance main body having no connection function to the wide area communication network 62. In this Embodiment, the household appliance 10 is equipped with the air conditioner 10-1, the microwave oven 10-2, the refrigerator 10-3, the washing machine 10-4, the air cleaner 10-5, and the rice cooker 10-6.

  The communication terminal device 30 can include a smartphone or a tablet terminal in which a dedicated application is downloaded. The communication terminal device 30 is equipped with sensors such as a camera, a position sensor that acquires position information (for example, GPS), a step sensor that measures the number of steps, a temperature sensor, and a humidity sensor.

  The cloud server 20 associates the communication terminal device 30 with the home appliance 10 and registers and manages the combination. By registering the combination in the cloud server 20, the communication terminal device 30 is used to remotely operate the associated home appliance 10 or to confirm the operating state of the remotely associated home appliance 10. Can do. Furthermore, the sensing information of the sensor provided in the communication terminal device 30 can be transmitted to the associated home appliance 10 to perform an operation using the sensing information. As described above, in the present embodiment, a captured image (sensing information) obtained by the camera (sensor) of the communication terminal device 30 is transmitted to the home appliance 10 for use.

  A plurality of home appliances 10 can be associated with one communication terminal device 30 as a target for remote operation. In addition, a plurality of communication terminal devices 30 can be associated as a remote operation source for one home appliance 10.

  The user home 50 is equipped with a wireless local area network (LAN) that is a narrow area communication network. The wireless LAN relay station 40 is connected to a wide area communication network 62 including the Internet. The relay station 40 is a communication device such as a WiFi (registered trademark) router or a WiFi access point. In the present embodiment, a configuration including the Internet is exemplified as the wide area communication network 62, but a telephone line network, a mobile communication network, a CATV (CAble TeleVision) communication network, a satellite communication network, or the like can also be used.

  The cloud server 20 and the home appliance 10 installed in the user home 50 can communicate with each other via the wide area communication network 62 and the wireless LAN relay station 40. Further, the cloud server 20 and the communication terminal device 30 can communicate with each other via the wide area communication network 62. The communication terminal device 30 and the Internet in the wide area communication network 62 are connected using 3G (3rd Generation), LTE (Long Term Evolution), a home or public WiFi access point, or the like. The home appliance 10 and the communication terminal device 30 are both wireless communication devices, and can communicate with each other via the relay station 40 without using the wide area communication network 62.

  Needless to say, the types and number of household appliances 10-1 to 10-6 illustrated in FIG. 1, the types and number of communication terminal devices 30-1 to 30-3, the number of user homes 50, and the like are merely examples. Similarly, only one cloud server 20 is described, but a configuration in which a plurality of cloud servers 20 share functions may be used. Moreover, although it demonstrates using the cloud server 20 which provides a cloud service as a server apparatus, it is not limited to a cloud server.

(Configuration of cloud server 20)
FIG. 2 is a block diagram illustrating a schematic configuration of the cloud server 20 in the remote operation system 100. The cloud server 20 is a server that manages each home appliance 10 provided in each user home 50, and includes a control unit 21, a storage unit 22, and a communication unit 23 as shown in FIG.

  The control unit 21 includes, for example, a computer device including an arithmetic processing unit such as a CPU (Central Processing Unit) and a dedicated processor, and controls the operation of each unit of the cloud server 20. The storage unit 22 stores various information (data) used in the cloud server 20. The communication unit 23 performs mutual communication with the home appliance 10 and the communication terminal device 30 via the wide area communication network 62.

  The control unit 21 includes an association management unit 21a, an authentication unit 21b, an information extraction unit (extraction unit) 21c, and an operation control unit (determination unit) 21d.

  The association management unit 21a manages the association between the communication terminal device 30 and the home appliance 10 that can be remotely operated. The storage unit 22 stores a correspondence table between the communication terminal device 30 and the home appliance 10 that is created and used by the association management unit 21a.

  The authentication unit 21b confirms whether there is any defect between the captured image transmitted from the communication terminal device 30 and the home appliance 10 that uses the captured image. Specifically, the presence / absence of defects is determined by corresponding / comparing the captured image transmitted from the communication terminal device 30 and information (such as a postal code, which will be described later) regarding the home appliance 10 registered for each home appliance 10. To do. The storage unit 22 also stores information about the home appliance 10 that the authentication unit 21b refers to for determination.

  Information on the shooting location is added to the image shot by the camera provided with the communication terminal device 30 together with information such as the shooting time. For the information on the shooting location, position information (for example, GPS) of the communication terminal device 30 at the time of shooting is used. Therefore, the authentication unit 21b compares the position information of the communication terminal device 30 at the time of transmitting the captured image or the information on the shooting location added to the captured image with the postal code that is one of the information about the target home appliance 10. Then, it is determined whether or not the photographed image is taken in the vicinity of the installation location of the target home appliance 10.

  The information extraction unit 21 c performs data processing on the sensing information transmitted from the communication terminal device 30 and extracts information for determining the operation content of the target home appliance 10. In the present embodiment, since the sensing information is a captured image, the information extraction unit 21c performs image processing, and extracts information for determining the operation content from the captured image. Specific examples are given below.

  1) If the captured image is an image of a room, the number of people included in the image is determined, and information on the number of people is extracted.

  2) If the captured image is an image of the outdoors, the weather (sunny, cloudy, rain, etc.) is determined from the image, and the weather information is extracted.

  3) If the captured image is an image of the washing tub of the washing machine, the amount of laundry in the laundry tub is determined, and information on the amount of laundry is extracted. For example, if the washing machine is a drum type, the amount of laundry can be roughly estimated by taking a picture of the drum containing the laundry from the front with the door open, and how much laundry is contained. Can be determined.

  4) When the photographed image is an image of the inside of the refrigerator, the ingredients in the warehouse are determined and information on the ingredients in the warehouse is extracted. The cloud server 20 side has a database on commercially available products (food) such as vegetables and fish meat, and by letting machine learning of images from various directions of each product, it is within the range shown in the image It is possible to recognize a certain amount of products (food ingredients).

  In addition, for example, information on the type of laundry may be extracted from a photographed image of a laundry tub by determining whether the laundry is a normal laundry or a large item such as a blanket or towel / sheet. For example, when the pattern / feature of the laundry in the image is uniform as a whole, it can be determined that the laundry is not a normal laundry but a large item such as a blanket, a towel, or a sheet.

  In addition, when food is stored in the refrigerator, it is necessary to store the food so that the expiration date of each food can be seen with the refrigerator door open, but from the captured image in the refrigerator, the food that is nearing the expiration date is determined. And you may extract the information of the foodstuff which is nearing the expiration date. The storage unit 22 also stores information necessary for the information extraction unit 21c to extract information for determining the operation content from the sensing information.

  The operation determination unit 21d determines the operation content to be executed by the target home appliance 10 based on the extraction result by the information extraction unit 21c. Specific examples are given below.

  I) When the target home appliance 10 is the air conditioner 10-1 and information on the number of people is extracted, the operation content corresponding to the number of people is determined. For example, when the number of people is larger than the standard, strong cooling is used during cooling, and weak heating is used during heating. On the other hand, when the number of people is less than the standard, the cooling is weak cooling and the heating is strong heating. Also, according to the change in the operation of the air conditioner 10-1, for example, “Today is busy, I'll cool down!” (When changing to strong cooling), “Today is busy, it ’s a little cooler! Also instruct the utterance such as “when changing to weak heating”.

  II) When the target home appliance 10 is the air conditioner 10-1 and the weather information is extracted, the operation content corresponding to the weather is determined. For example, when the weather is fine, the cooling is increased during cooling, and the heating is decreased during heating. On the other hand, when the weather is cloudy or rainy, a dehumidifying operation is added. Also, in accordance with the change in the operation of the air conditioner 10-1, for example, “Today is fine, let's cool! (When cooling is strengthened)” or “Today, it ’s mushy, dehumidify!” Utterances such as "when adding").

  III) When the target home appliance 10 is the washing machine 10-4 and information on the amount of laundry is extracted, the operation content corresponding to the amount of the selected item is determined. For example, when the amount is larger than the standard, the amount of detergent input is increased, and the washing, rinsing, dehydrating and drying times are lengthened. On the other hand, if the amount is less than the standard, the amount of detergent input is reduced to shorten the washing, rinsing, dehydration, and drying times. Also, in accordance with the change in the operation of the washing machine 10-4, for example, “Today is a heap, it takes a little longer! (When the amount is large)” or “Today is less, it ends soon!” Utterances such as “when the amount is small)” are also instructed.

  IV) When the target home appliance 10 is the washing machine 10-4 and the weather information is extracted, the operation content corresponding to the weather is determined. For example, when the weather is fine, the washing / drying mode is changed to the washing mode. On the other hand, when the weather is cloudy or rainy, the washing mode is set to the washing / drying mode. Also, in accordance with such a change in the operation of the washing machine 10-4, for example, “Today is fine, let's dry outside (when changing to the washing mode)” or “Today is hard to dry, dry Also tell the utterance “When you change to washing / drying mode”.

  V) When the target home appliance 10 is the refrigerator 10-3 or the microwave oven 10-2 and information on the food is extracted, the operation content suitable for the food is determined. The operation content suitable for the ingredients is, for example, an utterance action for proposing a cooking menu suitable for the ingredients.

  VI) When the target home appliance 10 is the air purifier 10-5 and the weather information is extracted, the operation content corresponding to the weather is determined. For example, when the weather is cloudy or rainy, the humidification operation is stopped. Also, in accordance with such an operation change of the air purifier 10-5, for example, an utterance such as “I am messing up today, stop humidification!” Is also instructed.

  In addition, for example, when information that the laundry is a blanket is extracted from the photographed image of the washing tub, when the information is changed to a washing course suitable for the blanket or information on ingredients that are close to the expiration date is extracted Various examples can be considered, such as suggesting a cooking menu that uses these ingredients.

  The voice data uttered by the home appliance 10 may be downloaded from the cloud server 20 to the home appliance 10, or if the content of the utterance can use the one provided in the home appliance 10, the utterance operation may be instructed. The storage unit 22 also includes information necessary for the operation determination unit 21d to determine the operation content to be executed by the home appliance 10 based on the information for determining the operation content extracted by the information extraction unit 21c. Stored.

  In addition, the operation determination unit 21d transmits a control command for driving the target home appliance 10 with the determined operation content (including the speech operation). The home appliance 10 that has received the control command operates according to the control command. The storage unit 22 also stores a correspondence table between operation contents and control commands.

(Configuration of communication terminal apparatus 30)
FIG. 3 is a block diagram illustrating a schematic configuration of the communication terminal device 30 in the home appliance network system 100. As shown in FIG. 3, the communication terminal device 30 includes a control unit 31, a storage unit 32, a display operation unit 33, a communication unit 34, a camera 36, a position sensor 37, a step sensor 38, a temperature sensor 35, and a humidity sensor 39. ing.

  For example, the control unit 31 includes a computer device that includes an arithmetic processing unit such as a CPU (Central Processing Unit) or a dedicated processor, and controls the operation of each unit of the communication terminal device 30. The control unit 31 controls the camera 36, the position sensor 37, the step number sensor 38, the temperature sensor 35, and the humidity sensor 39, and acquires a captured image, position information, and step number information as sensing information.

  The storage unit 32 stores various information (data) used in the communication terminal device 30. Images taken by the camera 36, step count information, and the like are also stored in the storage unit 32. The display operation unit 33 displays various types of information to be presented to the user, and accepts an operation input for the display information. The communication unit 34 performs communication via the wide area communication network 62 and mutual communication between devices via the wireless LAN relay station 40.

  In the control unit 31, a remote operation unit 31a and an information posting unit 31b are constructed. The remote operation unit 31a and the information posting unit 31b are constructed by a dedicated application or the like downloaded to the communication terminal device 30. The remote operation unit 31 a displays an operation screen corresponding to the home appliance 10 on the display operation unit 33 and accepts a remote operation on the home appliance 10. When the control unit 31 receives an operation instruction on the operation screen, the control unit 31 transmits to the cloud server 20 a control command that instructs execution of the received operation instruction.

  The information posting unit 31b receives and executes a process of transmitting the sensing information of the sensor included in the communication terminal device 30 itself to the cloud server 20 at a desired timing. The information posting unit 31b transmits sensing information from the sensor included in the communication terminal device 30 to the cloud server 20 together with information indicating the home appliance 10 that uses the sensing information. The information posting unit 31b displays a home appliance selection screen for selecting the home appliance 10 to which the sensing information is posted as shown in FIG. 4 on the display operation unit 33, and selects the home appliance 10 to post. In the example of FIG. 4, icons corresponding to the associated air conditioner 10-1, microwave oven 10-2, refrigerator 10-3, washing machine 10-4, and air purifier 10-5 are displayed. When the icon of home appliance 10 is tapped (operated) on the home appliance selection screen, information posting unit 31b determines home appliance 10 corresponding to the tapped icon as a posting destination of sensing information.

  Further, the information posting unit 31b displays an information selection screen for selecting sensing information to be posted to the home appliance 10 as illustrated in FIG. 5 on the display operation unit 33, and selects the sensing information to be posted. In the example of FIG. 5, icons corresponding to the camera 36, the position sensor 37, the step number sensor 38, the temperature sensor 35, and the humidity sensor 39 are displayed. When an icon of sensing information is tapped (operated) on the information selection screen, the information posting unit 31b determines the sensing information corresponding to the tapped icon as posted information.

  Then, when the position sensor 37 or the step sensor 38 is selected, the information posting unit 31b determines the posting information, and simultaneously transmits the position information or the step count information to the cloud server 20 together with the information indicating the home appliance 10 as the posting destination. To do. On the other hand, when the camera is selected, the camera 36 is activated, and when a transmission instruction is given after the image is captured, the captured image is transmitted to the cloud server 20 together with information indicating the home appliance 10 as a posting destination. Information on the shooting location is added to the captured image along with information on the shooting time.

(Configuration of home appliance 10)
The home appliance 10 is a device capable of network communication. The home appliance 10 receives various data including an operation signal (command) from the cloud server 20. Data received from the cloud server 20 includes data from the communication terminal device 30 transmitted via the cloud server 20 in addition to data from the cloud server 20 itself. Further, the home appliance 10 transmits information related to the home appliance 10 to the cloud server 20.

  FIG. 6 is a block diagram illustrating a schematic configuration of the home appliance 10. The home appliance 10 includes a control unit 11, a storage unit 13, a driving operation unit 14, a communication unit 15, and an audio output unit 16.

  The control part 11 consists of computer apparatuses comprised by arithmetic processing parts, such as CPU (Central Processing Unit) and a dedicated processor, for example, and controls operation | movement of each part of the household appliance 10. FIG. When the data received from the cloud server 20 is a control command for commanding the operation, the control unit 11 operates the driving operation unit 14 or speaks from the voice output unit 16 according to the received control command. The storage unit 13 stores various information used in the home appliance 10. The communication unit 15 performs mutual communication with the cloud server 20 via the wide area communication network 62.

  The driving operation unit 14 performs various driving operations according to the type of the home appliance 10 under the control of the control unit 11. For example, in the case of the air conditioner 10-1, there are a cooling operation, a heating operation, a dehumidifying operation, a blowing operation, and the like as operation modes, and these operation modes have strengths. In the case of the air purifier 10-5, there are an air cleaning operation mode, a humidification operation mode, an ion generation operation mode, and the like, and these operation modes have strengths. In the case of the refrigerator 10-3, there are a normal operation mode, a powerful operation mode, a high-speed ice making mode, and the like. Further, in the case of the washing machine 10-4, there are a washing course, a washing / drying course, a drying course, and the washing course includes details such as whether it is normal laundry, fashionable clothes, or blankets. I know. When it is the microwave oven 10-2, it is oven cooking and range cooking, and when it is the rice cooker 10-6, it is rice cooking.

  The audio output unit 16 includes, for example, a speaker and a driving unit thereof, and outputs a voice based on the audio data and speaks under the control of the control unit 11.

(Flow of processing to post sensing information to home appliances)
Processing of each device in the home appliance network system 100 will be described using a specific example of FIG. FIG. 7 shows processing of each device when a captured image is posted from the communication terminal device 30 to the air conditioner 10-1, and the air conditioner 10-1 performs an operation using the captured image.

  The communication terminal device 30 displays a home appliance selection screen (see FIG. 4) on the display operation unit 33 in accordance with a user operation (S1). When the air conditioner icon is tapped (operated) on the screen and a determination operation is performed, the air conditioner 10-1 is determined as the posting destination of the sensing information (S2). When the posting destination home appliance 10 is determined, an information selection screen (see FIG. 5) is displayed on the display operation unit 33 (S3). When a camera icon is tapped (operated) on the screen and a determination operation is performed, sensing information from the camera 36 is determined as transmission information (S4).

  When the transmission information is determined, the camera 36 is activated to cause the user to take an image, and when a transmission confirmation button or the like is tapped (operated), the captured image is displayed together with information indicating the post-destination air conditioner 10-1 together with the cloud server. 20 (S5).

  When the cloud server 20 receives the information indicating the posting destination air conditioner 10-1 and the photographed image from the communication terminal device 30 (S6), the cloud server 20 receives the positional information of the photographed image or the communication terminal device, and the air conditioner that is the posting destination. 10-10 is compared with postal code information registered in advance, and it is determined whether the image is taken near the air conditioner 10-1 (S7).

  In S7, when the image is captured near the air conditioner 10-1, the cloud server 20 performs data processing on the captured image, determines the number of people in the image or the weather from the image, and determines the operation content. Information for determination is extracted (S8). When the information on the number of people or the weather is extracted, the cloud server 20 determines the operation content to be executed by the air conditioner 10-1 according to the number of people or the weather (S9), and the air conditioner 10-1 determines the determined operation content. Is transmitted (S10).

  When the air conditioner 10-1 receives a control command from the cloud server 20 (S11), the air conditioner 10-1 performs an operation according to the control command (S12). For example, if there are many people included in the photographed image and a change to strong cooling or an utterance is commanded, the air conditioner 10-1 changes to strong heating and, for example, “Today is busy, cool! "When changing to").

  If it is determined in step S7 that the photographed image is not photographed near the air conditioner 10-1, the process does not proceed to S8. Thereby, even when the user selects an air conditioner that is different from the air conditioner 10-1 installed in the captured room when selecting the posting destination of the sensing information, such an error can be surely eliminated. In this case, information indicating a posting error may be returned to the communication terminal device 30 that is the transmission source of the captured image.

  As described above, in the home appliance network system 100 according to the present embodiment, the sensing information provided by the sensor included in the communication terminal device 30 is posted to the home appliance 10 from the communication terminal device 30, and the sensing information posted by the home appliance 10 is used. Perform the action.

Thereby, the sensing information which the household appliance 10 can utilize can be increased, without increasing a sensor for exclusive use in the inside of the household appliance 10, or its installation space, and the convenience of the household appliance 10 can be improved. 10-3
In the present embodiment, the procedure is to select the posting sensing information after selecting the posting home appliance 10, but the procedure to select the posting sensing information after selecting the posting sensing information. It is good. It is not necessary to be able to post the sensing information of all the mounted sensors in all home appliances 10 that can be selected as posting destinations, and the type of sensing information that can be posted may be determined according to the type of home appliance 10, The types of home appliances 10 that can be posted may be determined according to the type of sensing information. Moreover, although the authentication part 21b and the operation | movement determination part 21d were provided in the cloud server 20, it is good also as a structure provided in the household appliance 10. FIG.

[Embodiment 2]
In this Embodiment, in the said household appliance network system 100, the positional information (sensing information) by the position sensor 37 with which the communication terminal device 30 is provided is used as the air conditioner 10-1, the microwave oven 10-2, the washing machine 10-4, and the rice cooker. A configuration for posting and using in 10-6 will be described.

  When the sensing information transmitted from the communication terminal device 30 is position information, the authentication unit 21b in the cloud server 20 confirms whether or not there is any defect with the home appliance 10 that uses the position information. To skip.

  When the sensing information transmitted from the communication terminal device 30 is position information, the information extraction unit 21c determines the return time of the user who owns the communication terminal device 30 according to the position information, and returns home. Extract time information.

  The operation determination unit 21d determines the operation content to be executed by the target home appliance 10 based on the information of the return time extracted by the information extraction unit 21c. Specific examples are given below.

  I) When the target home appliance 10 is the air conditioner 10-1 and the information on the return time is extracted, the operation start time is adjusted, and when the user returns, the operation content that provides a comfortable room temperature is determined.

  II) When the target home appliance 10 is the washing machine 10-4 and the information on the return time is extracted, the operation start time of the reserved operation (for example, the laundry mode) is adjusted and completed when the user returns home. Determine the operation.

  III) When the target home appliance 10 is the microwave oven 10-2 or the rice cooker 10-6 and the information on the time of going home is extracted, the operation start time of the reserved operation (for example, oven cooking or rice cooking) is adjusted. Then, the operation content for cooking is determined when the user returns home.

[Embodiment 3]
In the present embodiment, in the home appliance network system 100, the step number information (sensing information) by the step number sensor 38 included in the communication terminal device 30 is posted to the microwave oven 10-2, the refrigerator 10-3, and the washing machine 10-4. The configuration to be used will be described.

  When the sensing information transmitted from the communication terminal device 30 is step count information, the authentication unit 21b in the cloud server 20 confirms whether there is no defect with the home appliance 10 that uses the step count information. To skip.

  When the sensing information transmitted from the communication terminal device 30 is step count information, the information extraction unit 21c determines the amount of exercise of the user who owns the communication terminal device 30 according to the step count information, Extract information.

  The operation determination unit 21d determines the operation content to be executed by the target home appliance 10 based on the information on the amount of exercise extracted by the information extraction unit 21c. Specific examples are given below.

  I) When the target home appliance 10 is the washing machine 10-4 and information on the amount of exercise is extracted, the operation content that matches the amount of exercise is determined. For example, the operation content to be uttered is determined such as “I wonder if there is laundry because I exercised a lot today.”

  II) When the target home appliance 10 is the microwave oven 10-2 or the refrigerator 10-3 and information on the amount of exercise is extracted, the operation content that matches the amount of exercise is determined. For example, the operation content to be uttered is determined such as “I have exercised a lot today, so take XXX! I recommend XXX”. ○○○ contains nutrients, etc., and xxx contains a cooking menu that includes XX nutrients.

[Embodiment 4]
In the present embodiment, in the home appliance network system 100, temperature and humidity information (sensing information) by the temperature sensor 35 and the humidity sensor 39 included in the communication terminal device 30 is converted into the microwave oven 10-2, the refrigerator 10-3, and the laundry. The structure which is posted and used for the machine 10-4 and the air purifier 10-5 will be described.

  When the sensing information transmitted from the communication terminal device 30 is temperature information or humidity information, the authentication unit 21b in the cloud server 20 checks whether there is any defect between the home appliance 10 that uses the information. Skip the process to do.

  When the sensing information transmitted from the communication terminal device 30 is temperature information or humidity information, the information extraction unit 21c has the communication terminal device 30 according to the temperature information or humidity information, but the space where the information extraction unit 21c is located. Temperature and humidity are determined, and temperature information or humidity information is extracted.

  The operation determination unit 21d determines the operation content to be executed by the target home appliance 10 based on the temperature information or the humidity information extracted by the information extraction unit 21c. Specific examples are given below.

  I) When the target home appliance 10 is the washing machine 10-4 and the temperature information is extracted, the operation content matching the temperature (air temperature) of the space in which the user is present is determined. For example, when the temperature is higher than the reference, in the washing mode, the drying time is shortened in anticipation of drying well when dried outside. In the drying mode, the drying time is shortened in anticipation of an increase in water temperature. Also, in response to such a change in the operation of the washing machine 10-4, for example, an utterance such as “Today is hot, it ends early! (When the temperature is high)” is also instructed.

  II) When the target home appliance 10 is the microwave oven 10-2 and the temperature information is extracted, the operation content suitable for the temperature (air temperature) of the space where the user is present is determined. For example, the operation content to be uttered is determined such as “Today is hot and cold XXX is recommended”. In xxx, there is a cooking menu according to the temperature.

  III) When the target home appliance 10 is the refrigerator 10-3 and the temperature information is extracted, the operation content suitable for the temperature (air temperature) of the space where the user is present is determined. For example, when the temperature is higher than the standard, it is predicted that a lot of ice is needed, and the high-speed ice making mode is changed. Also, in accordance with such an operation change of the refrigerator 10-3, for example, the operation content to be uttered is determined such as "It's hot today, let's make a lot of ice, put water in the ice making tank!"

  However, when the temperature sensor 35 and the humidity sensor 39 (particularly the temperature sensor 35) mounted on the communication terminal device 30 have sensing information that is not necessarily the temperature and humidity of the space where the user of the communication terminal device 30 is located. There is. This is for sensing the user's body temperature and the humidity near the user's palm when the user is holding the communication terminal device 30. Further, when the communication terminal device 30 is being charged, it is for sensing the temperature rise of the device itself being charged. Therefore, the temperature sensor 35 and the humidity sensor 39 are coupled with information such as an acceleration sensor mounted on the communication terminal device 30 so that the user does not hold the communication terminal device 30 or is not charging. It is necessary to post the sensing information.

[Embodiment 5]
In the home appliance network system 101 of the present embodiment, even if the home appliance 10 is not selected from the home appliance selection screen (see FIG. 4) in the home appliance network system 100, the cloud server 20 is based on the posted image information. The structure which determines the household appliance 10 of a posting destination is added.

  In the storage unit 22 of the cloud server 20, as information on the home appliance 10 used by the information extraction unit 21c, in addition to the zip code, the inside of the selection tub of the washing machine 10-4, the refrigerator 10-3, the air conditioner 10- Information for identifying the room in which 1 is installed by image discrimination is stored. The information extraction unit 21c performs image processing on the received captured image and determines whether the image is in the selected tub of the washing machine 10-4, the refrigerator 10-3, or the air conditioner 10-1.

  For example, if it determines with it being the image in the store | warehouse | chamber of refrigerator 10-3, the household appliance 10 of the posting destination of the said picked-up image will specify that it is refrigerator 10-3. Similarly, when it is determined that the image is in the washing tub of the washing machine 10-4, the home appliance 10 to which the photographed image is posted is specified as the washing machine 10-4. Moreover, if it determines with it being the room | chamber interior in which the air conditioner 10-1 is installed, the household appliance 10 of the posting destination of the said captured image will specify that it is the air conditioner 10-1.

  In addition, when the posting destination home appliance 10 cannot be specified, information indicating a posting error may be returned to the communication terminal device 30 that is the transmission source of the captured image.

  Processing of each device in the home appliance network system 101 will be described using a specific example of FIG. FIG. 8 shows that a photographed image is posted from the communication terminal device 30 to the cloud server 20, and the cloud server 20 specifies that the home appliance to be operated using the photographed image is the air conditioner 10-1. -1 shows the processing of each device when an operation using a captured image is performed.

  The communication terminal device 30 skips S1 and S2 in FIG. 7, and displays an information selection screen (see FIG. 5) on the display operation unit 33 according to the user's operation (S3). When a camera icon is tapped (operated) on the screen and a determination operation is performed, sensing information from the camera 36 is determined as transmission information (S4).

  When the transmission information is determined, the camera 36 is activated to cause the user to capture an image, and when a transmission confirmation button or the like is tapped (operated), the captured image (sensing information) is transmitted to the cloud server 20 (S21).

  When the cloud server 20 receives a photographed image from the communication terminal device 30 (S22), the cloud server 20 performs image processing on the received photographed image and specifies the posting home appliance 10 (S23). When the posting home appliance 10 is specified, the cloud server 20 performs data processing on the photographed image, determines the number of people in the image or the weather from the image, and extracts information for determining the operation content ( After that, the same processing as S9 to S12 in FIG. 7 is performed.

[Embodiment 6]
In the home appliance network system 102 according to the present embodiment, the cloud server 20 does not select the posting destination home appliance 10 from the home appliance selection screen (see FIG. 4) in the home appliance network system 100 based on the posted image information. Another configuration for determining the posting destination home appliance 10 is added.

  When the cloud server 20 receives sensing information from the communication terminal device 30, the information extraction unit 21c extracts data for data processing to determine the operation content of the target home appliance 10, and the operation determination unit 21d The operation content is determined based on the extracted information for all home appliances 10 that can be selected as posting destinations. The operation determining unit 21d uses a control command for driving the target home appliance 10 with the operation content (including the utterance operation) determined for each home appliance 10, and information on the date and time when the sensing information is posted from the communication terminal device 30. Send to home appliances together.

  Based on the received date and time information, the home appliance 10 receives the sensing information posted from the communication terminal device 30 when the door is opened or closed or the remote controller is operated. If it is determined that there is, an operation corresponding to the received control command is executed.

  The processing of each device in the home appliance network system 102 will be described using a specific example of FIG. 9 posts the sensing information from the communication terminal device 30 to the cloud server 20, and the cloud server 20 determines the operation content using the sensing information for all the home appliances 10 that can be selected as a posting destination, and the control command The processing of each device is shown when the air conditioner 10-1 determines that the posting is addressed to itself and causes only the air conditioner 10-1 to perform an operation using the captured image.

  The communication terminal device 30 skips S1 and S2 in FIG. 7, and displays an information selection screen (see FIG. 5) on the display operation unit 33 according to the user's operation (S3). On the screen, for example, when an icon of a step sensor is tapped (operated) and a determination operation is performed, step information (sensing information) is determined as transmission information (S4).

  When the transmission information is determined, the step count information is transmitted to the cloud server 20 (S25). When the cloud server 20 receives the number of steps information from the communication terminal device 30 (S26), the cloud server 20 determines the amount of exercise from the received number of steps information and extracts the amount of exercise information (S27). When the information on the amount of exercise is extracted, the cloud server 20 determines the operation content according to the amount of exercise for all the home appliances 10 that can be selected as a posting destination (S28), and controls the operation content determined for all the home appliances 10 The command is transmitted together with information on the date and time when the communication terminal device 30 transmitted the step count information (may be information regarding the date and time when the cloud server 20 received the step count information from the communication terminal device 30) (S29).

  All the home appliances 10 that have received the control command receive the sensing information posted from the communication terminal device 30 based on the date / time information and the presence / absence of user operation (opening / closing of the door or operation with the remote control) on the device itself. It is determined whether or not there is (S30), and if it is determined that it is addressed to itself, an operation corresponding to the received control command is executed (S31).

[Embodiment 7]
The home appliance network system 103 according to the present embodiment is capable of instructing the home appliance 10 to perform an operation instruction when sensing information from a sensor mounted on the communication terminal device 30 satisfies a certain condition in the home appliance network system 100. A configuration that pops up icons and buttons is added.

  As shown in FIG. 10, a special operation instruction unit 31 c capable of commanding the home appliance 10 with special operation contents that cannot be normally ordered is constructed in the control unit 31 of the communication terminal device 30. When the sensing information satisfies a preset condition, the special operation instruction unit 31c pops up an icon or a button that allows the home appliance 10 to command a special operation content that cannot be normally ordered.

  For example, when the sensing information is temperature information and the temperature (air temperature) exceeds 30 °, an icon that can command the air conditioner 10-1 to perform the special mode operation appears. In addition, when the sensing information is step count information, if the step count exceeds 10,000, a button for instructing the refrigerator 10-3 or the microwave oven 10-2 to propose a special cooking menu as shown in FIG. 11 pops up. Is displayed.

  When the icon displayed by the special operation instruction unit 31c is tapped (operated) and a determination operation is performed, the communication terminal device 30 sends an instruction to the cloud server 20 to suggest a special mode operation or a special cooking menu to the home appliance 10. The cloud server 20 determines the operation content and transmits a control command to the target home appliance 10 so that these are executed. The cloud server 20 creates audio data of the special cooking menu as necessary.

  As the special operation mode of the air conditioner 10-1, for example, modes such as “turbo cooling mode” and “full power cooling mode” that operate with the maximum operation amount of the compressor and the fan until the temperature set by the user can be considered. It is done.

  In addition, as a special operation (speech) of the refrigerator 10-3, for example, an introduction of a cooking menu that is effective in relieving fatigue such as “How did you exercise well? Conceivable. In addition, the cooking menu introduced here does not need to have the ingredients required for cooking.

  Also, as a special operation (speech) of the microwave oven 10-2, for example, “You have exercised well, prepared a fatigue recovery menu!” The cooking menu of pork ginger grilling may be downloaded from the cooking menu having a fatigue recovery effect stored in 20.

[Embodiment 8]
Corresponding management unit 21a, authentication unit 21b, information extraction unit 21c, operation determination unit 21d, remote operation unit 31a, information posting unit 31b, special operation in home appliance network systems 100 to 103 described in the first to seventh embodiments. Each of the instruction units 31c may be realized by a logic circuit (hardware) formed in an integrated circuit (IC chip) or the like, or may be realized by software using a CPU (Central Processing Unit).

  In the latter case, the communication terminal device 30, each home appliance 10, and the cloud server 20 in the home appliance network systems 100 to 103 each have a CPU that executes a program instruction that is software that implements each function, the program, and various data. A ROM (Read Only Memory) or a storage device (these are referred to as “recording media”) recorded so as to be readable by a computer (or CPU), a RAM (Random Access Memory) for expanding the program, and the like are provided. And the objective of this invention is achieved when a computer (or CPU) reads the said program from the said recording medium and runs it. As the recording medium, a “non-temporary tangible medium” such as a tape, a disk, a card, a semiconductor memory, a programmable logic circuit, or the like can be used. The program may be supplied to the computer via an arbitrary transmission medium (such as a communication network or a broadcast wave) that can transmit the program. The present invention can also be realized in the form of a data signal embedded in a carrier wave in which the program is embodied by electronic transmission.

  Moreover, the cloud server 20 is not necessarily required, and the home appliance may have the function possessed by the cloud server 20. Moreover, you may give a part of function of the cloud server 20 in embodiment to a household appliance.

  The present invention is not limited to the above-described embodiments, and various modifications are possible, and the present invention also relates to embodiments obtained by appropriately combining technical means disclosed in different embodiments. Is included in the technical scope. Furthermore, a new technical feature can be formed by combining the technical means disclosed in each embodiment.

[Summary]
The household appliance network system (100) which concerns on aspect 1 of this invention is a household appliance network system with which the household appliance (10), the server apparatus (cloud server 20), and the communication terminal device (30) were connected via the communication network (62). And the said communication terminal device is provided with the information posting part (31b) which receives and performs the process which transmits the sensing information of the sensor with which it is provided to the said server apparatus at a desired timing, The said server apparatus is the said communication An extraction unit (information extraction unit 21c) that extracts information for determining the operation content commanded to the home appliance from the sensing information received from the terminal device is provided, and the server device or the home appliance uses the information extracted by the extraction unit. A determination unit (operation determination unit 21d) for determining the operation content of the home appliance based on the operation determined by the determination unit And executes the contents.

  According to the said structure, the information posting part of a communication terminal device transmits the sensing information of the sensor with which a communication terminal device is provided to a server apparatus at a desired timing. The extraction part of a server apparatus extracts the information for determining the operation content which commands a household appliance from the sensing information received from the said communication terminal device. The determination unit determines the operation content of the home appliance based on the information extracted by the extraction unit, and the home appliance executes the operation content determined by the determination unit. Note that the determination unit can be mounted on either the server device or the home appliance.

  Thereby, the sensing information which a household appliance can utilize can be increased, and the convenience of a household appliance can be improved, without adding a dedicated sensor inside the household appliance or the installation space.

  In the home appliance network system (100) according to aspect 2 of the present invention, the information posting unit further displays an information selection screen for selecting a sensor to which sensing information is transmitted from a plurality of sensors provided. The sensing information of the sensor selected on the information selection screen is transmitted to the server device.

  According to the above configuration, by using the information selection screen, it is possible to easily select sensing information (target sensor) to be transmitted to the home appliance from a plurality of sensors mounted on the communication terminal device. .

  In the home appliance network system (100) according to aspect 3 of the present invention, the information posting unit further selects a home appliance to be used for sensing information from a plurality of home appliances that can use sensing information. It is the structure which displays a selection screen and transmits the information which shows the household appliance selected on this household appliance selection screen to the said server apparatus with the said sensing information.

  According to the said structure, it becomes possible by using a household appliance selection screen to select easily the household appliance of the object which transmits sensing information from the some household appliances which can utilize sensing information.

  In the home appliance network system (100) according to aspect 4 of the present invention, the sensing information is a captured image, position information, or step count information.

  In the home appliance network system (100) according to the aspect 5 of the present invention, the sensing information is a captured image, and the server device or the home appliance is related to information regarding a position of the captured image and a position of a home appliance registered in advance. It is the structure in which the certification | authentication part (21d) which compares information and confirms that there is no flaw in the household appliance selected on the said household appliance selection screen and a picked-up image is provided.

  According to the above configuration, the authentication unit compares the information related to the position of the captured image with the information related to the position of the home appliance registered in advance, and there is a discrepancy between the home appliance selected on the home appliance selection screen and the captured image. Check if there is any. Therefore, even if the user selects the home appliance to which the captured image is posted by mistake, the captured image is not used in the home appliance selected by mistake.

  The present invention relates to a home appliance network system in which a sensor having a communication function and a home appliance are connected via a communication network, and can be used for a remote operation system for remotely operating a home appliance using a communication terminal device.

DESCRIPTION OF SYMBOLS 10 Home appliance 11 Control part 14 Driving | operation operation part 16 Audio | voice output part 15 Communication part 20 Cloud server (server apparatus)
21 Control unit 21a Corresponding management unit 21b Authentication unit 21c Information extraction unit (extraction unit)
21d Operation determination unit (determination unit)
DESCRIPTION OF SYMBOLS 30 Communication terminal device 31 Control part 31a Remote operation part 31b Information contribution part 31c Special operation instruction part 33 Display operation part 35 Temperature sensor 36 Camera 37 Position sensor 38 Step sensor 39 Humidity sensor 100 Home appliance network system 101 Home appliance network system 102 Home appliance network system 103 Home Appliance Network System

Claims (4)

A home appliance network system in which a home appliance, a server device, and a communication terminal device are connected via a communication network,
The communication terminal device includes an information posting unit that receives and executes a process of transmitting sensing information of a sensor included in the communication terminal device to the server device at a desired timing,
The server device includes an extraction unit that extracts information for determining an operation content commanded to a home appliance from sensing information received from the communication terminal device,
The server device or the home appliance includes a determination unit that determines the operation content of the home appliance based on the information extracted by the extraction unit,
The home appliance executes the operation content determined by the determination unit,
The information posting unit displays an information selection screen for selecting a sensor to which sensing information is transmitted from a plurality of sensors included in the information posting unit, and displays sensing information of the sensor selected on the information selection screen. Transmitted to the server device, and
The information posting unit displays a home appliance selection screen for selecting a home appliance to be used for sensing information from a plurality of home appliances that can use sensing information, and selects the home appliance selected on the home appliance selection screen. The information which shows is transmitted to the said server apparatus with the said sensing information, The household appliance network system characterized by the above-mentioned . The home appliance network system according to claim 1 , wherein the sensing information is a captured image, position information, or step count information. The sensing information is a captured image;
The server device or the home appliance has no difference between the information about the position of the photographed image and the information about the position of the home appliance registered in advance and the home appliance selected on the home appliance selection screen and the photographed image. The home appliance network system according to claim 1 , further comprising an authorization unit for confirming. A home appliance network system in which a home appliance, a server device, and a communication terminal device are connected via a communication network,
The communication terminal device includes an information posting unit that receives and executes a process of transmitting sensing information of a sensor included in the communication terminal device to the server device at a desired timing,
The server device includes an extraction unit that extracts information for determining an operation content commanded to a home appliance from sensing information received from the communication terminal device,
The server device or the home appliance includes a determination unit that determines the operation content of the home appliance based on the information extracted by the extraction unit,
The home appliance executes the operation content determined by the determination unit,
The information posting unit displays a home appliance selection screen for selecting a home appliance to be used for sensing information from a plurality of home appliances that can use sensing information, and selects the home appliance selected on the home appliance selection screen. Information to be sent to the server device together with the sensing information,
The sensing information is a captured image;
The server device or the home appliance has no difference between the information about the position of the photographed image and the information about the position of the home appliance registered in advance and the home appliance selected on the home appliance selection screen and the photographed image. it characterized in that identification portion to confirm is provided home electric network system.

Images