JP7601721B2 - In-vehicle device and in-vehicle light emission control method - Google Patents

Description

The present invention relates to an in-vehicle device and an in-vehicle light emission control method.

One type of in-car device with audio functionality is provided with an audio knob for adjusting the output volume.

On the other hand, there are also in-vehicle devices that can perform functional operations for hands-free calling or voice recognition.

JP 2015-7873 A

The current state of a hands-free call (incoming call, answering, holding, outgoing call, etc.) is often notified to the user by displaying an image. However, when the hands-free call state is displayed as an image, the display of other necessary images (for example, the display of a navigation image) is prevented. In consideration of this, a control is being considered in which, by inputting a specific operation while the hands-free call state is displayed as an image, the state transitions (returns) to a state in which other necessary images (for example, a navigation image) are displayed (see Figure 16).

However, if this control causes a transition (return) to a state in which other necessary images (e.g. a navigation image) are displayed, the current state of the hands-free call becomes unclear or unclear to the user. If a dedicated indicator showing the current state of the hands-free call were provided, the current state of the hands-free call would be clearly communicated to the user, but the installation of a dedicated indicator would increase costs, etc., making it difficult to adopt. Although particular attention has been given to hands-free calls, the same circumstances also apply to voice recognition, etc.

The present invention aims to provide an in-vehicle device and an in-vehicle light emission control method that can effectively notify the status of functional operations.

The in-vehicle device of the present invention includes an audio knob for adjusting the output volume of an audio device installed in a vehicle, a light-emitting unit for illuminating the audio knob by emitting light, and a main control unit for controlling the execution of an audio function operation for reproducing an output sound signal of a tuner unit of the audio device, and another function operation different from the audio function operation, and the main control unit causes the light-emitting unit to emit light according to a first control when the audio function operation is executed, and causes the light-emitting unit to emit light according to a second control different from the first control when the other function operation is executed.

The present invention makes it possible to provide an in-vehicle device and an in-vehicle light emission control method that can effectively notify the status of functional operations.

1 is an overall configuration diagram of an in-vehicle system according to an embodiment of the present invention; 1 is a schematic top view of a vehicle according to an embodiment of the present invention. 1 is a diagram showing a state of a vehicle interior of a vehicle according to an embodiment of the present invention; 1 is a front view of a multimedia device according to an embodiment of the present invention; 11A and 11B are diagrams illustrating modified examples of the relationship between an audio knob and a light-emitting portion in a multimedia device according to an embodiment of the present invention. 2 is a diagram showing a navigation image that may be displayed on the display screen of FIG. 1; 4 is a flowchart showing the operation of a multimedia device according to a first embodiment of the present invention. FIG. 11 is an explanatory diagram of light emission control related to audio operation in the first example belonging to an embodiment of the present invention. 11A to 11C are diagrams showing examples (three examples) of a display image and a light-emitting state of a light-emitting unit during execution of a hands-free function operation according to a first example belonging to an embodiment of the present invention. FIG. 10 is an explanatory diagram of state transition during execution of a voice recognition function operation according to a first embodiment of the present invention. 10A to 10C are diagrams showing examples (three examples) of a display image and a light-emitting state of a light-emitting unit during execution of a voice recognition function operation according to a first example belonging to an embodiment of the present invention. FIG. 11 is a diagram showing how a display image is changed during execution of a hands-free function operation according to a second embodiment of the present invention. FIG. 11 is a diagram showing how a display image is changed during execution of a voice recognition function operation according to a third example of an embodiment of the present invention. FIG. 13 is a diagram showing how a display image is changed during execution of a hands-free function operation according to a fourth embodiment of the present invention. FIG. 13 is a diagram showing how a display image is changed during execution of a voice recognition function operation according to a fifth example of the present invention. 11A to 11C are diagrams showing changes in the display content on the display screen of an in-vehicle device according to the reference technology;

Below, examples of embodiments of the present invention will be described in detail with reference to the drawings. In each of the drawings referred to, the same parts are given the same reference numerals, and duplicated explanations of the same parts will be omitted as a general rule. In this specification, for the sake of simplicity, a symbol or code referring to information, a signal, a physical quantity, a member, etc. may be written, and the name of the information, signal, physical quantity, member, etc. corresponding to the symbol or code may be omitted or abbreviated. For example, the multimedia device referred to by "1" described below (see FIG. 1) may be written as multimedia device 1 or abbreviated as device 1, but they all refer to the same thing.

Figure 1 shows the overall configuration of an in-vehicle system SYS according to an embodiment of the present invention. The in-vehicle system SYS comprises a multimedia device 1, a camera unit 2, a GPS processing unit 3, an in-vehicle sensor unit 4, and an in-vehicle network 5. Also shown in Figure 1 is a terminal device TM and a communication network NET. Figure 2 is a schematic top view of a vehicle CR equipped with the in-vehicle system SYS. Figure 3 shows a part of the interior of the vehicle CR. In the interior of the vehicle CR, a steering wheel CC1, a shift lever CC2, and an ignition switch CC3 are provided near the driver's seat as parts operated by the driver, and the multimedia device 1 is also installed.

The steering wheel CC1 is a ring-shaped part that is rotated by the driver to adjust the direction of travel of the vehicle CR in response to the rotation. The shift lever CC2, based on the driver's operation, sets whether the vehicle CR can proceed and the direction of travel of the vehicle CR, and changes the gears of the vehicle CR. The ignition switch CC3, based on the driver's operation, specifies whether or not power is supplied to each electrical device of the vehicle CR, and specifies whether or not the power source (engine, etc.) of the vehicle CR is started or stopped.

The multimedia device 1, the camera unit 2, the GPS processing unit 3, and the in-vehicle sensor unit 4 are connected to an in-vehicle network 5 formed inside the vehicle CR, and can communicate any information and signals bidirectionally with each other via the in-vehicle network 5, or can communicate any information and signals unidirectionally. A CAN (Controller Area Network) or the like can be used as the in-vehicle network 5. Note that in FIG. 1, only the multimedia device 1, the camera unit 2, the GPS processing unit 3, and the in-vehicle sensor unit 4 are shown as components connected to the in-vehicle network 5, but various other devices (e.g., a power steering system, a brake system, a radar device, a drive recorder) that are not shown in the figure may also be connected to the in-vehicle network 5.

The vehicle CR is a vehicle (such as an automobile) that can travel on a road surface. However, the vehicle CR may be any type of vehicle. In the vehicle CR, the direction from the driver's seat toward the steering wheel CC1 is defined as "forward," and the direction from the steering wheel CC1 toward the driver's seat is defined as "rearward." The direction perpendicular to the front-to-rear direction and parallel to the road surface is defined as the left-to-right direction. Left and right in the left-to-right direction are defined as left and right as seen by the driver of the vehicle CR, who is sitting in the driver's seat of the vehicle CR and facing forward.

The camera unit 2 is composed of one or more cameras installed in the vehicle CR, and each camera installed in the camera unit 2 is called a unit camera. Each unit camera is equipped with an imaging element such as a CMOS (Complementary Metal Oxide Semiconductor) image sensor or a CCD (Charge-Coupled Device) image sensor and an optical system, and captures images within its own field of view (in other words, its shooting area) and generates image information indicating the image obtained by the capture (i.e., the image within the field of view). The image information generated by the unit camera is hereinafter referred to as camera image information, and the two-dimensional image represented by the camera image information is referred to as a camera image. Each unit camera periodically and repeatedly captures images within its own field of view at a predetermined shooting frame rate. The camera image information generated by the unit camera is sent sequentially to the multimedia device 1.

The camera unit 2 is provided with a camera 2R as one of the unit cameras (see FIG. 2). The camera 2R is a rear camera that has a field of view outside the vehicle CR and in the rear area of the vehicle CR, and is installed in an appropriate position on the body of the vehicle CR. Although it is not clear from FIG. 2, the depression angle of the camera 2R is appropriately set so that the road surface outside the vehicle CR and within a specified area to the rear of the vehicle CR is included in the field of view of the camera 2R.

In addition to the camera 2R as a rear camera, the camera unit 2 may also be provided with a front camera that is outside the vehicle CR and has a field of view in the forward area of the vehicle CR, a right camera that is outside the vehicle CR and has a field of view in the right area of the vehicle CR, and a left camera that is outside the vehicle CR and has a field of view in the left area of the vehicle CR. Each of the front camera, right camera, and left camera is also a type of unit camera.

The GPS processing unit 3 detects the position (current location) of the vehicle CR by receiving signals from multiple GPS satellites that form the GPS (Global Positioning System), and generates vehicle position information indicating the detected position. The position of the vehicle CR means the location where the vehicle CR is present. In the vehicle position information, the position (current location) of the vehicle CR is expressed by longitude and latitude on the earth. The vehicle position information is generated sequentially at a predetermined cycle, and the generated vehicle position information is sent sequentially to the multimedia device 1.

The vehicle sensor unit 4 is made up of multiple vehicle sensors installed in the vehicle CR, and generates vehicle sensor information using each of the vehicle sensors. The vehicle sensor information is generated sequentially at a predetermined cycle, and the generated vehicle sensor information is sent sequentially to the multimedia device 1. Figure 1 shows a shift lever sensor 4A and a vehicle speed sensor 4B as two sensors included in the multiple sensors.

The shift lever sensor 4A is a sensor that detects the position of the shift lever CC2 of the vehicle CR, and is configured using a displacement sensor, etc. The position of the shift lever CC2 is set to one of multiple positions including drive position, reverse position, and parking position based on the driver's operation. When the shift lever CC2 is in the drive position, the vehicle CR moves forward (travels forward) in response to depression of the accelerator pedal of the vehicle CR, and when the shift lever CC2 is in the reverse position, the vehicle CR moves backward (travels backward) in response to depression of the accelerator pedal of the vehicle CR. When the shift lever CC2 is in the parking position, the vehicle CR does not move forward or backward regardless of depression of the accelerator pedal of the vehicle CR. Information indicating the position of the shift lever CC2 detected by the shift lever sensor 4A is called shift information. The shift information is included in the on-board sensor information.

The vehicle speed sensor 4B detects the traveling speed (movement speed) of the vehicle CR. Information indicating the traveling speed of the vehicle CR detected by the vehicle speed sensor 4B is referred to as vehicle speed information. The vehicle speed information is included in the on-board sensor information. In addition, the on-board sensor information includes accelerator information indicating the amount of depression of an accelerator pedal provided on the vehicle CR, brake information indicating the amount of depression of a brake pedal provided on the vehicle CR, and steering angle information indicating the steering angle of the steering wheel CC1.

The configuration of the multimedia device 1 will be described. The multimedia device 1 comprises a main control unit 10, a memory unit 20, a display screen 30, an operation unit 40, a microphone 50, a light-emitting unit 60, a communication module 70, and an audio device 80. However, the audio device 80 may be a device provided outside the multimedia device 1.

The main control unit 10 includes a CPU (Central Processing Unit), RAM (Random Access Memory), ROM (Read Only Memory), etc., and provides overall control over the overall operation of the multimedia device 1.

The memory unit 20 is made up of a magnetic disk or semiconductor memory, or a combination of these, and stores and keeps track of various programs and data. The CPU of the main control unit 10 executes the programs in the memory unit 20 to realize various functions.

The display screen 30 consists of a liquid crystal display panel or an organic electroluminescence panel, etc., and displays any image under the control of the main control unit 10.

The operation unit 40 accepts various operations from the user of the multimedia device 1. The contents of the operations input to the operation unit 40 are transmitted to the main control unit 10. The user of the multimedia device 1 (hereinafter simply referred to as the user) is an occupant of the vehicle CR. An occupant other than the driver of the vehicle CR can also be a user, but in the following, it is assumed that the driver of the vehicle CR is the user. The operation unit 40 is provided with an audio knob 41. A touch panel may be configured on the display screen 30, in which case the touch panel is included as a component of the operation unit 40.

The microphone 50 picks up sounds around the microphone 50, converts the picked up sounds into an audio signal, which is an electrical signal, and outputs it. The sound that the microphone 50 picks up is mainly the voice uttered by the user. The microphone 50 is installed in a position where it is easy to pick up the voice uttered by the user (here, the driver) (for example, a specified position on the steering wheel CC1 or a specified position near the driver's seat).

The light-emitting unit 60 emits light under the control of the main control unit 10. More specifically, the light-emitting unit 60 is composed of a light-emitting element that emits light by converting an electrical signal supplied from the main control unit 10 into an optical signal. The light-emitting element is, for example, a light-emitting diode. However, the light-emitting element may be any element that emits light. The light-emitting unit 60 may emit light by organic electroluminescence. The light-emitting unit 60 may be formed by combining a liquid crystal and a backlight. The light-emitting unit 60 may function as a light source that illuminates the audio knob 41 by emitting light. In this case, the light-emitting unit 60 may illuminate the audio knob 41 from the outside, or may illuminate the audio knob 41 from the inside so that the audio knob 41 itself appears to be emitting light. The light-emitting unit 60 may be configured to emit light in only one color, but in the following, the light-emitting unit 60 is assumed to be configured to emit light in multiple colors.

The communication module 70 realizes wireless two-way communication with an external device for the multimedia device 1. The external device includes a terminal device TM. The terminal device TM is a terminal device owned by the user of the multimedia device 1 (here, the driver). The terminal device TM is, for example, a mobile phone (including a smartphone) or a tablet terminal. However, it is assumed that the terminal device TM has a telephone function. The communication module 70 and the terminal device TM are wirelessly connected to each other by a short-range wireless communication line such as Bluetooth (registered trademark). The communication network NET includes a mobile communication network having a mobile communication base station and the Internet. The communication module 70 is wirelessly connected to the communication network NET, and any other device (server device, etc.) connected to the Internet is also included in the above-mentioned external device.

The audio device 80 includes a tuner unit 81, an audio amplifier 82, and a speaker 83. The speaker 83 may be a speaker externally connected to the multimedia device 1 (i.e., a speaker provided outside the multimedia device 1 and connected to the multimedia device 1). The tuner unit 81 outputs an audio signal acquired from a predetermined sound source to the audio amplifier 82. Specifically, for example, the tuner unit 81 acquires an audio signal such as a piece of music from a music recording medium stored in or connected to the audio device 80, or acquires an audio signal such as a radio program from a broadcast wave received from a broadcast station, and outputs the acquired audio signal to the audio amplifier 82. The audio amplifier 82 can execute an audio function operation that amplifies the audio signal output from the tuner unit 81 and supplies the acquired audio signal (an amplified signal of the output audio signal of the tuner unit 81) to the speaker 83. When the audio function operation is executed, the audio signal output from the tuner unit 81 is reproduced using the speaker 83 (output as sound from the speaker 83).

Figure 4 shows an example of the appearance of the multimedia device 1 when viewed from the front. In the front view of the multimedia device 1, the device 1 has a roughly rectangular main housing, in which the display screen 30 is fitted at a predetermined position, and the audio knob 41 and the light-emitting unit 60 are installed at a position offset from the fitting position of the external screen 30. The audio knob 41 (hereinafter may be abbreviated as the knob 41) is a knob-shaped operating part used to adjust and set the output volume of the audio device 80, and protrudes from the surface of the main housing of the multimedia device 1. A user can adjust and set the output volume of the audio device 80 by pinching and rotating the knob 41 with his/her fingers. The output volume of the audio device 80 (hereinafter may be simply referred to as the output volume) is an amount proportional to the volume of the sound output from the speaker 83, and the volume of the sound is larger as the output volume is larger. The multimedia device 1 is installed near the driver's seat of the vehicle CR so that the user of the multimedia device 1 can easily view the display screen 30 and can easily operate the touch panel and knob 41 on the display screen 30.

The light-emitting unit 60 is provided in correspondence with the knob 41. The knob 41 is a cylindrical knob that protrudes from the surface of the main housing of the multimedia device 1 in a direction perpendicular to the surface. Here, the light-emitting unit 60 is formed by one or more light-emitting elements arranged in a position surrounding the cylindrical housing of the knob 41. In FIG. 4, the hatched area corresponds to the light-emitting unit 60. However, the shape of the light-emitting unit 60 is not limited to the example in FIG. 4. For example, the light-emitting unit 60 may be a linear light-emitting unit 60_1 arranged adjacent to the knob 41 as shown in FIG. 5(a), or a circular light-emitting unit 60_2 installed on the upper surface of the cylindrical housing of the knob 41 as shown in FIG. 5(b). In addition, the knob 41 and the light-emitting unit 60 may be installed in a housing (not shown) other than the main housing.

Referring again to FIG. 1, the main control unit 10 includes a navigation unit 11, a hands-free support unit 12, a voice recognition support unit 13, and a light emission control unit 14 as functional blocks realized by the CPU of the main control unit 10 executing a program in the storage unit 20.

The navigation unit 11 executes a predetermined navigation operation based on the vehicle position information. The navigation operation includes a process of displaying a navigation image on the display screen 30. Image 600 in FIG. 6 is an example of a navigation image. The navigation image 600 includes a map image showing the current position of the vehicle CR and a map of the area around the current position of the vehicle CR. In the navigation image 600, an icon indicating the current position of the vehicle CR and a route to a destination specified and set by the user are superimposed on the map image. The navigation operation includes a process of guiding the vehicle CR along the route.

The hands-free compatible unit 12 (hereinafter referred to as the HF compatible unit 12) executes a hands-free function operation (hereinafter referred to as the HF function operation) that enables hands-free calling. In this embodiment, a hands-free call is a hands-free call made by a user of the multimedia device 1, and the hands-free call is made possible by the HF compatible unit 12 working in cooperation with the terminal device TM, the communication module 70, the microphone 50, and the speaker 83.

A description of hands-free calling will now be given. For the sake of concreteness and clarity of explanation, the user of the multimedia device 1 will be referred to as the first person, and a person located outside the vehicle CR who is different from the user of the multimedia device 1 will be referred to as the second person. A terminal device (such as a mobile phone including a smartphone) used by the second person that is different from the terminal device TM will be referred to as the second terminal device. Hereinafter, the terminal device TM may be referred to as the first terminal device. In a hands-free call, the first person can talk to the second person by phone using the microphone 50 and a speaker installed in the vehicle CR. Here, the speaker installed in the vehicle CR used for the hands-free call is speaker 83. However, a hands-free call may also be realized using another speaker (not shown) installed in the vehicle CR.

The second terminal device corresponds to the other party's device for the first terminal device (TM). When a second person uses the second terminal device to call the first terminal device (TM) over a telephone line, an incoming call signal is received by the first terminal device (TM) through the communication network NET. When the first terminal device (TM) receives the incoming call signal, it transmits an incoming call notification signal to the multimedia device 1. The incoming call signal and the incoming call notification signal contain identification information associated with the second terminal device (e.g., the telephone number of the second terminal device, etc.).

When the multimedia device 1 receives an incoming call notification signal from the first terminal device (TM), the HF compatible unit 12 starts executing the HF function operation. Here, there are multiple states of the HF function operation, including the call receiving state (states other than the call receiving state will be described later). The call receiving state may be read as the response state. In the call receiving state, a voice signal indicating the contents of the second person's speech is input from the first terminal device (TM) to the HF compatible unit 12 through the communication module 70, and the HF compatible unit 12 outputs the voice signal indicating the contents of the second person's speech from the speaker 83. In the call receiving state, a voice signal indicating the contents of the first person's speech (i.e., the contents of the speech of the user of the multimedia device 1) is input from the microphone 50 to the HF compatible unit 12, and the HF compatible unit 12 transmits the voice signal indicating the contents of the first person's speech to the first terminal device (TM) through the communication module 70. The first terminal device (TM) transmits the audio signal (audio signal indicating the contents of the first person's speech) received from the multimedia device 1 to the communication network NET including the base station and the mobile communication network. As a result, the audio signal indicating the contents of the first person's speech is transmitted to the second terminal device, and the audio signal is reproduced by the second terminal device. Although the above describes a hands-free call when the second terminal device calls the first terminal device (TM), it is also possible to make a hands-free call after calling the second terminal device from the first terminal device (TM).

The voice recognition support unit 13 (hereinafter referred to as the VR support unit 13) executes voice recognition function operation (hereinafter referred to as the VR function operation). The VR function operation includes a voice recognition process that recognizes the contents of the user's speech, and a response process that responds to the contents of the user's speech using the recognition result of the voice recognition process. A voice signal representing the contents of the user's speech picked up by the microphone 50 is particularly referred to as a user speech voice signal. During the user's speech period, the user speech voice signal is output from the microphone 50. In the voice recognition process, the contents of the user's speech are converted into text data based on the user speech voice signal, and the contents of the user's speech are recognized and understood. In the response process, an appropriate response corresponding to the contents of the user's speech is made based on the results of the voice recognition process. The VR function operation can be realized by software-based artificial intelligence.

The light emission control unit 14 controls the light emission of the light emission unit 60. The control of light emission includes control of whether the light emission unit 60 emits light or not, as well as control of the light emission mode of the light emission unit 60. The content of the light emission control of the light emission unit 60 depends on whether the HF function operation or the VR function operation is currently being executed (details will be described later).

Below, several specific operation examples, application techniques, modification techniques, etc. related to the in-vehicle system SYS will be described in multiple examples. The matters described above in this embodiment are applied to each of the following examples unless otherwise specified and unless there is a contradiction. If there are any matters in each example that contradict the matters described above, the description in each example may take precedence. Furthermore, unless there is a contradiction, the matters described in any of the multiple examples shown below can also be applied to any other example (i.e., any two or more of the multiple examples can also be combined).

<<First Example>>
A first embodiment will be described with reference to Fig. 7. The flow of operations of the multimedia device 1 will be described with attention to the light emission control of the light emitting unit 60, the HF function operation, and the VR function operation.

Let us consider step S11 as the starting point. In step S11, the multimedia device 10 is in an initial state. In the initial state, neither the HF function operation nor the VR function operation is being performed. Also, it is assumed that the terminal device TM and the multimedia device 1 are set so that the HF function operation is performed when the terminal device TM receives an incoming call signal (however, this does not include the case where incoming call rejection is set).

In step S12 following step S11, the navigation operation is started. After the navigation operation is started in step S12, it can be understood that the navigation image 600 as shown in FIG. 6 continues to be displayed on the display screen 30 unless otherwise specified.

After step S12, the process proceeds to step S13. In step S13, the main control unit 10 (specifically, the HF support unit 12) judges whether or not an incoming call notification signal has been received from the terminal device TM. If an incoming call notification signal has been received (Y in step S13), the process proceeds to step S31. If an incoming call notification signal has not been received (N in step S13), the process proceeds to step S14. In step S14, the main control unit 10 (specifically, the VR support unit 13) judges whether or not a predetermined voice recognition activation condition has been met. If the voice recognition activation condition has been met (Y in step S14), the process proceeds to step S51. If the voice recognition activation condition has not been met (N in step S14), the process proceeds to step S15. The voice recognition activation condition is met, for example, when the user speaks a predetermined wake-up keyword, and a user-uttered voice signal indicating the wake-up keyword is input from the microphone 50 to the VR support unit 13. Alternatively, for example, the voice recognition activation condition may be met when a predetermined wake-up operation is input to the operation unit 40.

When proceeding to step S15, both the HF function operation and the VR function operation are in a non-executing state. Also, it is assumed that a predetermined operation instructing the execution of the above-mentioned audio function operation has been input to the operation unit 40 in step S11 or step S15. Then, in step S15, the main control unit 10 controls the audio device 80 so that the audio function operation is executed, and as a result, the audio signal output from the tuner unit 81 is output as sound from the speaker 83.

The light emission control unit 14 executes the light emission control _CNT1 in step S15. The light emission mode of the light emission unit 60 is controlled in the light emission control _CNT1 . For example, in the light emission control _CNT1 , the light emission unit 60 is caused to emit light in a light emission mode according to the volume setting value VOL _AD . The volume setting value VOL _AD represents a setting value of the output volume of the audio device 80 in the audio function operation. Therefore, in the audio function operation, the output volume of the audio device 80 becomes a volume according to the volume setting value VOL _AD . When the audio function operation is being performed, the user can arbitrarily adjust the volume setting value VOL _AD by operating the knob 41. The volume setting value VOL _AD is adjusted in multiple stages.

An example of the light emission control _CNT1 is shown in Fig. 8. In the example of Fig. 8, only the hatched area of the entire area of the light emitting unit 60 indicates the part that emits light. In the example of Fig. 8, the light emitting unit ₆₀ is configured to be able to change its own light emitting area in multiple stages, and the light emission control unit 14 increases the light emitting area of the light emitting unit 60 as the volume setting value VOL _AD increases. The light emission control CNT1 may be a control that causes the light emitting unit 60 to emit light at a constant level regardless of the volume setting value VOL _AD . Note that if the audio function operation is stopped when the process reaches step S15, the light emission control unit 14 may cause the light emitting unit 60 to completely not emit light. After step S15, the process returns to step S13.

In step S31, the HF support unit 12 determines whether or not incoming call rejection is set. The user can set or unset incoming call rejection by operating the operation unit 40. In step S31, if incoming call rejection is set (Y in step S31), the process returns to step S13, whereas if incoming call rejection is not set (N in step S31), the process proceeds to step S32.

In step S32, the HF compatible unit 12 activates the HF function operation (i.e., starts the execution of the HF function operation). Also, when the process reaches step S32, the light emission control unit 14 sets the light emission control of the light emitting unit 60 to the light emission control CNT _2A , and thereafter, the light emission mode of the light emitting unit 60 is controlled by the light emission control CNT _{2A until the HF function operation ends. The light emission mode of the light emitting unit 60 is controlled by the light emission control CNT 2A. The light emission control CNT 2A is different} from the above-mentioned light emission control CNT ₁ and the light emission control CNT _2B described later (see steps S15 and S51). In the light emission control CNT _2A , the light emitting unit 60 emits light in one of a plurality of different light emission modes according to the state of the HF function operation, and the plurality of light emission modes according to the state of the HF function operation include light emission modes LM _{2A_1} , LM _{2A_2} , and LM _{2A_3} described later (see FIG. 9 (a) to (c)).

After the HF function operation is started, the state of the HF function operation (hereinafter referred to as the HF state) becomes one of a number of states including an incoming call state, an answering state, and a hold state. When an incoming call notification signal is received, the HF state is an incoming call state before the user inputs a predetermined answering instruction operation to the operation unit 40, and when the user then inputs a predetermined answering instruction operation to the operation unit 40, the HF state switches from the incoming call state to the answering state. When the HF state is an answering state, the connection state of the telephone line between the first terminal device (TM) and the second terminal device is maintained, and a telephone conversation (hands-free conversation for the first person) between the first person (user of the multimedia device 1) and the second person is possible as described above.

After the HF state transitions to the receive state, when the user inputs a predetermined hold instruction operation to the operation unit 40, the HF state transitions to the receive state. In the hold state, the exchange of audio signals of the first and second persons' speech between the first terminal device (TM) and the second terminal device is stopped while maintaining the connection of the telephone line between the first terminal device (TM) and the second terminal device. Therefore, in the hold state, the output of the audio signal indicating the contents of the sound picked up by the microphone of the second terminal device (including the contents of the sound picked up by the microphone 50 (including the contents of the sound picked up by the first person) from the speaker 83 is stopped, and the transmission of the audio signal indicating the contents of the sound picked up by the microphone 50 (including the contents of the sound picked up by the first person) to the second terminal device is stopped. When the HF state is in the hold state, if the user inputs a predetermined hold release operation to the operation unit 40, the HF state returns to the receive state.

However, when the HF state is in the call receiving state or the call on hold state, if the user inputs a predetermined call end instruction operation to the operation unit 40, or if a predetermined call end instruction operation is input to the operation unit (not shown) of the second terminal device, the telephone line between the first terminal device (TM) and the second terminal device is disconnected. The disconnection of the telephone line is called "call end."

In step S33 following step S32, the main control unit 10 executes an incoming call notification process for notifying the user that the HF state is an incoming call state (there is an incoming call at the terminal device TM). In the incoming call notification process, a predetermined incoming call image 611 is displayed on the display screen 30 as shown in FIG. 9(a). Therefore, when the process proceeds from step S11 to step S33 via steps S13, S31, and S32, the display image on the display screen 30 is switched from the navigation image 600 to the incoming call image 611. The incoming call image 611 is an image indicating that there is an incoming call at the terminal device TM. Furthermore, in the incoming call notification process, the light emitting unit 60 emits light in a predetermined light emitting mode LM _{2A_1} as shown in FIG. 9(a). After step S33, the process proceeds to step S34.

In step S34, the HF support unit 12 checks the HF state. If the HF state is an incoming call state, the process returns to step S33, but if the HF state has switched from an incoming call state to a call receiving state, the process proceeds to step S35.

In step S35, the main control unit 10 executes a call receiving notification process to notify the user that the HF state is a call receiving state (a call is being received using the terminal device TM). In the call receiving notification process, a predetermined call receiving image 612 is displayed on the display screen 30 as shown in FIG. 9B. Therefore, when the HF state transitions from the incoming call state to the call receiving state, the display image on the display screen 30 is switched from the incoming call image 611 to the call receiving image 612. The call receiving image 612 is an image indicating that the HF state is in the call receiving state. Furthermore, in the call receiving notification process, the light emitting unit 60 emits light in a predetermined light emitting mode LM _{2A_2} as shown in FIG. 9B. After step S35, the process proceeds to step S36. In the call receiving state, the speech content of the second person is output from the speaker 83, but when the user operates the knob 41 in the call receiving state, the output volume of the speaker 83 in the call receiving state is adjusted. When the knob 41 is operated to adjust the output volume of the speaker 83 in the call receiving state, the light emission control unit 14 may temporarily change the light emission mode of the light-emitting unit 60 from the light emission mode _{LM2A_2} (for example, the light emission mode of the light-emitting unit 60 may be temporarily changed to a light emission mode corresponding to the output volume of the speaker 83).

In step S36, the HF support unit 12 checks the HF state. If the HF state is the call receiving state, the process returns to step S35. However, if the HF state has switched from the call receiving state to the hold state, the process proceeds to step S37. Also, if the call has been ended in step S36, the process proceeds to step S39.

In step S37, the main control unit 10 executes a hold notification process to notify the user that the HF state is on hold. In the hold notification process, a predetermined hold image 613 is displayed on the display screen 30 as shown in FIG. 9C. Therefore, when the HF state transitions from the call receiving state to the call on hold state, the display image on the display screen 30 is switched from the call receiving image 612 to the call on hold image 613. The call on hold image 613 is an image indicating that the HF state is on hold. Furthermore, in the hold notification process, the light emitting unit 60 emits light in a predetermined light emitting mode LM _{2A_3} as shown in FIG. 9C. After step S37, the process proceeds to step S38.

In step S38, the HF support unit 12 checks the HF state. If the HF state is a hold state (i.e., if the hold state continues), the process returns to step S37. If the hold state is released and the HF state transitions from a hold state to a call receiving state, the process returns to step S35. Also, if the call has been ended in step S38, the process proceeds to step S39.

In step S39, the HF support unit 12 ends the HF function operation (i.e., ends the execution of the HF function operation). Also in step S39, the light emission control unit 14 ends (stops) the light emission control CNT _2A for the light emission unit 60. After step S39, the process returns to step S13. Therefore, when the light emission control CNT _2A ends in step S39 after step S15, the process returns to step S15 via step S13, whereby the light emission unit 60 returns to a state in which it is illuminated by the light emission control CNT ₁ (assuming, however, that a new incoming call notification signal has not been received and that the voice recognition activation condition has not been satisfied).

Although it is not necessarily clear from Fig. 9 (a) to (c), the light emission modes LM _{2A_1} , LM _{2A_2} , and LM _{2A_3} are different from each other. As long as the light emission modes LM _{2A_1} , LM _{2A_2} , and LM _{2A_3} are different from each other, their light emission modes are arbitrary. The light emission modes can be differentiated by varying the light emission color, the presence or absence of blinking, the blinking frequency, and the movement of the light-emitting part. For example, in the light emission mode LM _{2A_1} , the entire area of the light-emitting unit 60 blinks at high speed with a red light emission color, in the light emission mode LM _{2A_2} , the entire area of the light-emitting unit 60 blinks at low speed with a green light emission color, and in the light emission mode LM _{2A_3} , the light-emitting unit 60 rotates and lights up with a blue light emission color. The fast blinking refers to repeating a unit blinking operation at a predetermined first frequency, in which the light-emitting element of the light-emitting unit 60 is turned on and then turned off, and the slow blinking refers to repeating the unit blinking operation at a predetermined second frequency lower than the first frequency. The rotating lighting refers to a light-emitting mode in which the light-emitting part of the light-emitting unit 60 appears to rotate around a predetermined axis (a light-emitting mode in which the light-emitting part is changed sequentially so that the light-emitting part changes around a predetermined axis). The images 611 to 613 in Fig. 9 (a) to (c) are also different from each other. The light-emitting unit 60 is illuminated in the light-emitting modes LM _{2A_1} , LM _{2A_2} , and LM _{2A_3} , and has a function of notifying the user that the current HF state is an incoming call state, an answering state, and a hold state, respectively. The light emission modes LM _{2A_ 1} , LM _{2A_ 2} and LM _{2A_ 3} may all be different from the light emission mode (see FIG. 8) of the light-emitting unit 60 when the light-emission control CNT ₁ is executed in step S15 in FIG.

The flow of operations when moving from step S14 to step S51 will be described. In step S51, the VR support unit 13 activates the VR function operation (i.e., starts the execution of the VR function operation). In addition, when it reaches step S51, the light emission control unit 14 sets the light emission control of the light emission unit 60 to the light emission control CNT _2B , and thereafter, the light emission mode of the light emission unit 60 is controlled by the light emission control CNT _2B until the VR function operation ends. The light emission mode of the light emission unit 60 is controlled by the light emission control CNT _2B . In the light emission control CNT _2B , the light emission unit 60 emits light in one of a plurality of different light emission modes according to the state of the VR function operation, and the plurality of light emission modes according to the state of the VR function operation include light emission modes LM _{2B_1} , LM _{2B_2} , and LM _{2B_3} described later (see FIG. 11 (a) to (c)).

After the VR function operation is started, the state of the VR function operation (hereinafter referred to as the VR state) becomes one of several states including a speech waiting state, a speech input state, and a response state. The transition between these states is shown in FIG. 10. When the VR function operation is started, the VR state first becomes a speech waiting state waiting for the user to start speaking. The period during which the user speaks after the VR function operation is started is referred to as the user speech period. During the user speech period, the VR state is a speech input state. The speech input state corresponds to a state in which the content of the user's speech is being input to the microphone 50. The VR support unit 13 may determine the start timing and end timing of the user speech period based on the strength and spectrum of the output audio signal of the microphone 50, etc. When the user's speech is interrupted for a certain period of time after the start of the user speech period (for example, when the strength of the signal component of the human voice in the output audio signal of the microphone 50 remains below a predetermined value for a certain period of time), the VR support unit 13 can determine that the user speech period has ended. When the user speech period ends, the VR state transitions from the speech input state to the response state. In the response state, a voice recognition process that recognizes the contents of the user's speech during the user speech period and a response process that responds to the contents of the user's speech using the recognition results of the voice recognition process are executed. Note that the voice recognition process can be started midway through the user speech period.

Depending on the content of the user's speech during the user speech period, there are cases where the VR function operation ends after the response process is completed, and cases where the VR state returns to the speech standby state after the response process is completed. The former case corresponds to the transition from step S57 to step S58 in FIG. 7, and the latter case corresponds to the return from step S57 to step S52 in FIG. 7.

Returning to the description of the process flow after step S51, referring to FIG. 7, in step S52 following step S51, the main control unit 10 executes speech standby notification process for notifying the user that the VR state is in a speech standby state. In the speech standby notification process, a predetermined standby image 621 is displayed on the display screen 30 as shown in FIG. 11(a). Therefore, when the process proceeds from step S11 to step S52 via steps S12 to S14 and S51, the display image on the display screen 30 is switched from the navigation image 600 to the standby image 621. The standby image 621 is an image indicating that the user's speech is being waited for. Furthermore, in the speech standby notification process, the light emitting unit 60 emits light in a predetermined light emitting mode LM _{2B_1} as shown in FIG. 11(a). After step S52, the process proceeds to step S53.

In step S53, the VR support unit 13 checks the VR state. If the VR state is a speech standby state, the process returns to step S52, but if the VR state has switched from a speech standby state to a speech input state, the process proceeds to step S54. After moving from step S51 to step S52, if the user starts speaking, the process proceeds to step S54.

In step S54, the main control unit 10 executes a speech input notification process to notify the user that the VR state is in a speech input state. In the speech input notification process, a predetermined input image 622 is displayed on the display screen 30 as shown in FIG. 11(b). Therefore, when the VR state transitions from the speech waiting state to the speech input state, the display image on the display screen 30 is switched from the waiting image 621 to the input image 622. The input image 622 is an image indicating that the VR state is in a speech input state. Furthermore, in the speech input notification process, the light emitting unit 60 emits light in a predetermined light emitting mode LM _{2B_2} as shown in FIG. 11(b). After step S54, the process proceeds to step S55.

In step S55, the VR support unit 13 checks the VR state. If the VR state is in the speech input state, the process returns to step S54, but if the VR state has switched from the speech input state to the support state, the process proceeds to step S56.

In step S56, the main control unit 10 executes an in-service notification process to notify the user that the VR state is in a service-in-progress state. In the in-service notification process, a predetermined in-service image 623 is displayed on the display screen 30 as shown in FIG. 11(c). Therefore, when the VR state transitions from the speech input state to the in-service state, the display image on the display screen 30 switches from the input image 622 to the in-service image 623. The in-service image 623 is an image indicating that the VR state is in a service-in-progress state. Furthermore, in the in-service notification process, the light-emitting unit 60 emits light in a predetermined light-emitting manner LM _{2B_3} as shown in FIG. 11(c).

In step S56, while the in-service notification process is being executed, the VR support unit 13 executes a voice recognition process that recognizes the contents of the user's speech during the user's speech period, and a response process that responds to the contents of the user's speech using the recognition result of the voice recognition process. When the response process is completed, the in-service notification process is stopped and the process proceeds to step S57. The in-service notification process may also be stopped at the stage where the response process is started.

In step S57, the VR support unit 13 judges whether or not the VR function operation should be continued. If it is judged that the VR function operation should be continued (Y in step S57), the process returns to step S52, whereas if not (N in step S57), the process proceeds to step S58.

The judgment process in step S57 depends on the contents of the user's speech during the user's speech period. For example, consider the following first case. In the first case, during the user's speech period, the user makes an utterance instructing to increase the audio volume setting value VOL _AD . In the first case, the VR support unit 13 increases the volume setting value VOL _AD by a certain value in the response process in step S56. In the first case, since the user's request is satisfied by increasing the volume setting value VOL _AD , the VR support unit 13 judges that the VR function operation may be terminated (N in step S57) and proceeds to step S58.

For example, consider the second case in which the user's utterance during the user utterance period is an utterance instructing to set a nearby restaurant as the destination of the vehicle CR. In the response process of step S56, the VR support unit 13 in the second case searches for restaurants in the vicinity of the vehicle CR through the communication module 70 and the communication network NET while referring to the vehicle position information. Here, it is assumed that the first to third restaurants are found as restaurants in the vicinity of the vehicle CR. Then, in the response process of step S56, the VR support unit 13 in the second case notifies the user to inquire which of the first to third restaurants to set as the destination of the vehicle CR while showing the user information on the first to third restaurants (the notification is performed using the speaker 83, or using the display screen 30 and the speaker 83). In such a second case, since a response from the user is required after the notification of the above inquiry, it is determined that the VR function operation should be continued (Y in step S57), and the process returns to step S52.

In step S58, the VR support unit 13 ends the VR function operation (i.e., ends the execution of the VR function operation). Also in step S58, the light emission control unit 14 ends (stops) the light emission control CNT _2B for the light emission unit 60. After step S58, the process returns to step S13. Therefore, when the light emission control CNT _2B ends in step S58 after step S15, the process returns to step S15 via step S13, whereby the light emission unit 60 returns to a state in which it is illuminated by the light emission control CNT ₁ (assuming, however, that the reception of an incoming call notification signal and the new voice recognition activation condition are not satisfied).

Although it is not necessarily clear from Figs. 11(a) to (c), the light emission modes LM _{2B_1} , LM _{2B_2} , and LM _{2B_3} are different from each other. As long as the light emission modes LM _{2B_1} , LM _{2B_2} , and LM _{2B_3} are different from each other, their light emission modes are arbitrary. The light emission modes can be differentiated by varying the light emission color, the presence or absence of blinking, the frequency of blinking, and the movement of the light emission point. For example, in the light emission mode LM _{2B_1} , the entire area of the light emission unit 60 blinks slowly with a green light emission color, in the light emission mode LM _{2B_2} , the entire area of the light emission unit 60 blinks quickly with a red light emission color, and in the light emission mode LM _{2B_3} , the light emission unit 60 rotates and lights up with a blue light emission color. The meanings of slow blinking, fast blinking, and rotating lighting are as described above. Note that the images 621 to 623 in Figs. 11(a) to (c) are also different from each other. The light emission of the light-emitting unit 60 in the light-emitting modes _{LM2B_1} , _{LM2B_2} , and _{LM2B_3} has a function of notifying the user that the current VR state is a speech waiting state, a speech input state, and a response state, respectively. The light-emitting modes _{LM2B_1} , _{LM2B_2} , and _{LM2B_3} may all be different from the light-emitting mode of the light-emitting unit 60 when the light-emitting control _CNT1 is executed in step S15 of FIG. 7 (see FIG. 8).

As described above, the light emission control unit 14 causes the light emitting unit 60 to emit light according to the light emission control CNT ₁ when the audio function operation is performed, and causes the light emitting unit 60 to emit light according to the other light emission control (CNT _2A , CNT _2B ) when the other function operation (HF function operation, VR function operation) is performed. This makes it possible to notify the user of information related to the execution of the other function operation (whether or not the other function operation is being executed, etc.) using the light emitting unit 60 for the knob 41 originally provided for the audio function operation (thus, without adding a new member). During the execution period of the other function operation, even if the display image of the display screen 30 is switched from the image for the other function operation, the light emitting unit 60 emits light according to the other light emission control (CNT _2A , CNT _2B ), so that the user can recognize information related to the execution of the other function operation (whether or not the other function operation is being executed, etc.).

<<Second Example>>
A second embodiment will now be described. The second embodiment is based on the first embodiment, and the matters described in the first embodiment are also applied to the second embodiment.

As described in the first embodiment, when the main control unit 10 executes an HF function operation, it displays a predetermined image corresponding to the HF function operation on the display image 30. Examples of predetermined images corresponding to the HF function operation include an incoming call image 611, an incoming call image 612, and an on-hold image 613 (see Figures 9(a) to (c)). Here, referring to Figure 12, the period from time t11 to time t13 is the execution period of the HF function operation. Time t12 is a time after time t11 and before time t13, and the display change condition is met at time t12 (in detail, the display change condition is not met until time t12, and is met at time t12). The execution period of the HF function operation corresponds to the period from the transition to step S32 in Figure 7 until step S39. During the execution period of the HF function operation, when a predetermined display change condition is satisfied (time t12) while a predetermined image (611, 612, or 613) corresponding to the HF function operation is displayed on the display screen 30, the main control unit 10 changes the display image on the display screen 30 from the predetermined image corresponding to the HF function operation to a predetermined destination image. In the second embodiment, the display change condition is the first display change condition, and the destination image is the navigation image 600. After the first display change condition is satisfied, the display of the incoming call image 611, the answering call image 612, and the on-hold image 613 by the processes of steps S33, S35, and S37 in FIG. 7 is stopped (see FIGS. 9(a) to (c)), and the display image on the display screen 30 becomes the navigation image 600.

On the other hand, the light emission control unit 14 that controls the light emission of the light-emitting unit 60 maintains the light emission control for the light-emitting unit 60 to the light emission control CNT _2A during the execution period of the HF function operation. Therefore, during the execution period of the HF function operation, the light emission control unit 14 causes the light-emitting unit 60 to continue to emit light in accordance with the light emission control CNT _2A even after the display image on the display screen 30 is changed from a predetermined image corresponding to the HF function operation to a change destination image in response to the establishment of the display change condition (see step S32 in FIG. 7).

As a result, even if the specified image corresponding to the HF function operation is no longer displayed, the light emission mode of the light-emitting unit 60 can inform the user of the current HF state (state of HF function operation).

When a navigation operation accompanied by the display of the navigation image 600 is being continuously performed, an incoming call to the terminal device TM may cause a transition to a period in which an HF function operation is being performed. During the period in which an HF function operation is being performed, a predetermined image (611, 612, or 613) corresponding to the HF function operation is displayed, thereby allowing the user to know the current HF state, but at this time, the navigation image 600 is prevented from being viewed.

The user can input a specific navigation return instruction to the multimedia device 1 at any timing during the execution period of the HF function operation. For example, a navigation return instruction is input to the multimedia device 1 by inputting a specific navigation return operation to the operation unit 40. When a navigation return instruction is input to the multimedia device 1 during the execution period of the HF function operation, a first display change condition, which is a display change condition according to the second embodiment, is met. When the first display change condition is met, the main control unit 10 changes the display image on the display screen 30 from a specific image corresponding to the HF function operation to the navigation image 600. The navigation return instruction corresponds to an instruction to make this change.

By adopting such a configuration, during the execution of the HF function operation, the state of the display screen 30 can be returned to a state in which the navigation image 600 is displayed, according to the user's wishes (it is possible to shorten the period during which the navigation image 600 is not displayed).

12, the display image on the display screen 30 returns from the call receiving image 612 to the navigation image 600, so that the driver as the user can thereafter view the navigation image 600, but the light emission control CNT _2A of the light-emitting unit 60 continues during the execution period of the HF function operation (see step S32 in FIG. 7). Therefore, during the execution period of the HF function operation, if the HF state (state of the HF function operation) is the same during the period (first period) during which the call receiving image 612 is displayed on the display screen 30 and the period (second period) during which the navigation image 600 is displayed on the display screen 30, the light-emitting control unit 14 causes the light-emitting unit 60 to emit light in the same light-emitting mode during the former period and the latter period. In the example of FIG. 12, the display image on the display screen 30 is changed from the call receiving image 612 to the navigation image 600 at time t12. However, since the HF state does not change from the call receiving state at time t12, the light-emitting unit 60 emits light in the same light-emitting manner _{LM2A_2} throughout the former period and the latter period.

The same applies when the display image on the display screen 30 is changed from the incoming call image 611 or the hold image 613 to the navigation image 600 at time t12. Furthermore, as described above, since the light emission patterns LM _{2A_1} to LM _{2A_3} are different from one another, the light emission of the light-emitting unit 60 according to the light emission patterns LM _{2A_1} to LM _{2A_3} has the function of notifying the user of the current HF state.

Therefore, even if a predetermined image corresponding to the HF function operation (in the example of FIG. 12, the call receiving image 612) is no longer displayed in response to the establishment of the display change condition, it is possible to inform the user of the current HF state (state of the HF function operation) by the light emission mode of the light-emitting unit 60. In other words, while being provided with the navigation image 600, the user can check the current HF state (whether the HF state immediately before time t12 is continuing, etc.) by looking at the light-emitting unit 60 as necessary.

Note that, although the input of a navigation return operation by the user to the multimedia device 1 has been described as the first display change condition in the second embodiment, the first display change condition, which is the display change condition in the second embodiment, may be any condition. For example, during the execution period of an HF function operation, the first display change condition may be satisfied when a predetermined time (e.g., 5 seconds) has elapsed since a predetermined image (611, 612, or 613) corresponding to the HF function operation started to be displayed on the display screen 30.

<<Third Example>>
A third embodiment will now be described. The third embodiment is based on the first embodiment, and the matters described in the first embodiment are also applied to the third embodiment.

As described in the first embodiment, when the main control unit 10 executes a VR function operation, it displays a predetermined image corresponding to the VR function operation on the display image 30. Examples of predetermined images corresponding to the VR function operation include a standby image 621, an input image 622, and a corresponding image 623 (see Figures 11 (a) to (c)). Here, referring to Figure 13, the period from time t21 to time t23 is the execution period of the VR function operation. Time t22 is a time after time t21 and before time t23, and the display change condition is met at time t22 (in detail, the display change condition is not met until time t22, and is met at time t22). The execution period of the VR function operation corresponds to the period from the transition to step S51 in Figure 7 until step S58. During the execution period of the VR function operation, when a predetermined display change condition is satisfied (time t22) while a predetermined image (621, 622, or 623) corresponding to the VR function operation is displayed on the display screen 30, the main control unit 10 changes the display image on the display screen 30 from the predetermined image corresponding to the VR function operation to a predetermined destination image. In the third embodiment, the display change condition is the first display change condition, and the destination image is the navigation image 600. After the first display change condition is satisfied, the display of the standby image 621, the input image 622, and the corresponding image 623 by the processes of steps S52, S54, and S56 in FIG. 7 is stopped (see FIGS. 11(a) to (c)), and the display image on the display screen 30 becomes the navigation image 600.

On the other hand, the light emission control unit 14 that controls the light emission of the light-emitting unit 60 maintains the light emission control for the light-emitting unit 60 to the light emission control CNT _2B during the execution period of the VR function operation. Therefore, during the execution period of the VR function operation, the light emission control unit 14 continues the light emission of the light-emitting unit 60 in accordance with the light emission control CNT _2B even after the display image on the display screen 30 is changed from a predetermined image corresponding to the VR function operation to a change destination image in response to the establishment of the display change condition (see step S51 in FIG. 7).

As a result, even if the specified image corresponding to the VR function operation is no longer displayed, the light emission mode of the light-emitting unit 60 can inform the user of the current VR state (the state of the VR function operation).

When a navigation operation involving the display of the navigation image 600 is being continuously performed, a transition may occur to a period during which a VR function operation is being performed. During the period during which the VR function operation is being performed, a specific image (621, 622, or 623) corresponding to the VR function operation is displayed, thereby allowing the user to know the current VR state, but at this time, the navigation image 600 is prevented from being viewed.

The user can input a specific navigation return instruction to the multimedia device 1 at any timing during the execution period of the VR function operation. The navigation return instruction is as described in the second embodiment. When a navigation return instruction is input to the multimedia device 1 during the execution period of the VR function operation, the first display change condition, which is the display change condition according to the third embodiment, is met. When the first display change condition is met, the main control unit 10 changes the display image on the display screen 30 from a specific image corresponding to the VR function operation to the navigation image 600. The navigation return instruction corresponds to an instruction to make this change.

By adopting such a configuration, during the execution period of the VR function operation, the state of the display screen 30 can be returned to a state in which the navigation image 600 is displayed according to the user's wishes, and the period during which the navigation image 600 is not displayed can be shortened.

13, the display image on the display screen 30 returns from the standby image 621 to the navigation image 600, so that the driver as the user can thereafter view the navigation image 600, but the light emission control CNT _2B of the light-emitting unit 60 continues during the execution period of the VR function operation (see step S51 in FIG. 7). Therefore, if the VR state (state of the VR function operation) is the same during the execution period of the VR function operation, during the period (first period) during which the standby image 621 is displayed on the display screen 30 and during the period (second period) during which the navigation image 600 is displayed on the display screen 30, the light-emitting control unit 14 causes the light-emitting unit 60 to emit light in the same light-emitting mode during the former period and the latter period. In the example of FIG. 13, the display image on the display screen 30 is changed from the standby image 621 to the navigation image 600 at time t22. However, since the VR state does not change from the speech standby state at time t22, the light-emitting unit 60 emits light in the same light-emitting manner _{LM2B_1} throughout the former and latter periods.

The same applies when, at time t22, the image displayed on the display screen 30 is changed from the input image 622 or the corresponding image 623 to the navigation image 600. Furthermore, as described above, since the light emission patterns LM _{2B_1} to LM _{2B_3} are different from one another, the light emission of the light-emitting unit 60 according to the light emission patterns LM _{2B_1} to LM _{2B_3} has the function of notifying the user of the current VR state.

Therefore, even if a predetermined image (standby image 621 in the example of FIG. 13) corresponding to the VR function operation is no longer displayed in response to the establishment of the display change condition, it is possible to inform the user of the current VR state (status of the VR function operation) by the light emission mode of the light-emitting unit 60. In other words, while being provided with the navigation image 600, the user can check the current VR state (whether the VR state immediately before time t22 is continuing, etc.) by looking at the light-emitting unit 60 as necessary.

Note that while the user's input of a navigation return operation to the multimedia device 1 has been described as the first display change condition in the third embodiment, the first display change condition, which is the display change condition in the third embodiment, may be any condition. For example, during the execution period of a VR function operation, the first display change condition may be satisfied when a predetermined time (e.g., 5 seconds) has elapsed since a predetermined image (621, 622, or 623) corresponding to the VR function operation started to be displayed on the display screen 30.

<<Fourth Example>>
A fourth embodiment will be described. In the fourth embodiment, a modified technique of the second embodiment will be described. The description of the second embodiment also applies to the fourth embodiment unless otherwise specified.

Refer to Figures 12 and 14. In the second embodiment corresponding to Figure 12, the display change condition is the first display change condition and the destination image is a navigation image 600, whereas in the fourth embodiment corresponding to Figure 14, the display change condition is the second display change condition and the destination image is a rear camera image 630. The rear camera image 630 is a camera image captured by camera 2R (rear camera; see Figure 2).

The second display change condition is met when the vehicle CR is set in the reverse direction. That is, the second display change condition is met when the position of the shift lever CC2 matches the reverse position. The main control unit 10 can determine whether the second display change condition is met based on the shift information. When the second display change condition is met during the execution period of the HF function operation, the main control unit 10 changes the display image on the display screen 30 from a predetermined image corresponding to the HF function operation to a rear camera image 630. The predetermined image corresponding to the HF function operation may be any of the incoming call image 611, the call receiving image 612, and the call on hold image 613, but FIG. 14 shows the call receiving image 612 as an example of the predetermined image. After the second display change condition is met, the display of the incoming call image 611, the incoming call image 612, and the on-hold image 613 resulting from the processing of steps S33, S35, and S37 in FIG. 7 is stopped (see FIGS. 9(a)-(c)), and the image displayed on the display screen 30 becomes the rear camera image 630.

According to the fourth embodiment, when it becomes necessary to display the rear camera image 630 during the execution of the HF function operation, it is possible to quickly transition to a state in which the rear camera image 630 is displayed.

In this case, as in the second embodiment, if the HF state (state of the HF function operation) is the same during the period during which the call receiving image 612 is displayed on the display screen 30 (first period) and the period during which the rear camera image 630 is displayed on the display screen 30 (second period) during the execution period of the HF function operation, the light emission control unit 14 causes the light emitting unit 60 to emit light in the same light emission mode during the former period and the latter period. In the example of FIG. 14, the display image on the display screen 30 is changed from the call receiving image 612 to the rear camera image 630 at the boundary of time t12, but since the HF state does not change from the call receiving state at the boundary of time t12, the light emitting unit 60 emits light in the same light emission mode LM _{2A_2} over the former period and the latter period. Therefore, the user can check the current HF state (whether the HF state immediately before time t12 is continuing, etc.) by looking at the light emitting unit 60 as necessary while receiving the rear camera image 630.

<<Fifth Example>>
A fifth embodiment will be described. In the fifth embodiment, a modified technique for the third embodiment will be described. The description of the third embodiment also applies to the fifth embodiment unless otherwise specified. Similar to the modification of the second embodiment (FIG. 12) to the fourth embodiment (FIG. 14), the fifth embodiment is a modification of the third embodiment (FIG. 13).

Refer to Figures 13 and 15. In the third embodiment corresponding to Figure 13, the display change condition is the first display change condition and the destination image is the navigation image 600, whereas in the fifth embodiment corresponding to Figure 15, the display change condition is the second display change condition and the destination image is the rear camera image 630.

The rear camera image 630 and the second display change condition are as described in the fourth embodiment. When the second display change condition is satisfied during the execution period of the VR function operation, the main control unit 10 changes the display image on the display screen 30 from the predetermined image corresponding to the VR function operation to the rear camera image 630. The predetermined image corresponding to the VR function operation may be any of the standby image 621, the input image 622, and the corresponding image 623, but FIG. 15 shows the standby image 621 as an example of the predetermined image. After the second display change condition is satisfied, the display of the standby image 621, the input image 622, and the corresponding image 623 by the processing of steps S52, S54, and S56 in FIG. 7 is stopped (see FIGS. 11(a) to (c)), and the display image on the display screen 30 becomes the rear camera image 630.

According to the fifth embodiment, when it becomes necessary to display the rear camera image 630 during the execution period of the VR function operation, it is possible to quickly transition to a state in which the rear camera image 630 is displayed.

In this case, as in the third embodiment, if the VR state (state of the VR function operation) is the same during the period during which the standby image 621 is displayed on the display screen 30 (first period) and the period during which the rear camera image 630 is displayed on the display screen 30 (second period) during the execution period of the VR function operation, the light emission control unit 14 causes the light emission unit 60 to emit light in the same light emission mode during the former period and the latter period. In the example of FIG. 15, the display image on the display screen 30 is changed from the standby image 621 to the rear camera image 630 at the boundary of time t22, but since the VR state does not change from the speech standby state at the boundary of time t22, the light emission unit 60 emits light in the same light emission mode LM _{2B_1} over the former period and the latter period. Therefore, the user can check the current VR state (whether the VR state immediately before time t22 is continuing, etc.) by watching the light emission unit 60 as necessary while receiving the rear camera image 630.

<<Sixth Example>>
A sixth embodiment will now be described. In the sixth embodiment, some supplementary matters, applied techniques, modified techniques, and the like to the matters described above will be described.

The flowchart in FIG. 7 focuses on a hands-free call when the second terminal device calls the first terminal device (TM), but the same operations and controls as described above are possible when a hands-free call is made after the first terminal device (TM) calls the second terminal device. For example, when the first terminal device (TM) calls the second terminal device in HF function operation, the user inputs a predetermined calling operation to the operation unit 40. In response to the input of the calling operation, the HF compatible unit 12 causes the first terminal device (TM) to transmit a calling signal toward the communication network NET. When an incoming call signal based on the calling signal is received by the second terminal device and an answering operation is input at the second terminal device, a hands-free call is possible.

After the operation for making a call is input to the operation unit 40, the HF state is in the calling state until the operation for receiving a call is input to the second terminal device. The state in which the operation for receiving a call is input to the second terminal device and a hands-free call is possible is the calling state. The calling state may be considered to be the same as the receiving state described above. In the calling state, the main control unit 10 executes a calling notification process for notifying the user that the HF state is in the calling state. In the calling notification process, a predetermined calling image (not shown) is displayed on the display screen 30 and the light-emitting unit 60 is caused to emit light in a predetermined light-emitting state LM _{2A_4} (not shown). The light-emitting state LM _{2A_4} has a function of notifying the user that the HF state is in the calling state. The light-emitting state LM _{2A_4} may be different from any of the light-emitting states LM _{2A_1} to LM _{2A_3} .

The light emission mode _{LM2A_4} is one of the light emission modes of the light-emitting unit 60 during the execution period of the HF function operation, and is a light emission mode according to the light emission control _CNT2A (see step S32 in FIG. 7). Therefore, even if the display change condition is satisfied while the light-emitting unit 60 is emitting light in the light-emitting mode _{LM2A_4} during the execution period of the HF function operation, and the display image on the display screen 30 is changed from the transmission image to the change destination image (navigation image 600 or rear camera image 630), the light-emitting unit 60 continues to emit light in the light-emitting mode _{LM2A_4} as long as the HF state is maintained as the transmission state.

The multimedia device 1 in FIG. 1 is an example of an in-vehicle device according to the present invention. The multimedia device 1 itself can be considered to correspond to the in-vehicle device according to the present invention. Alternatively, it is possible to form an in-vehicle device having only some of the components of the multimedia device 1. In this case, it is preferable that the in-vehicle device has at least a knob 41, a light-emitting unit 60, and a main control unit 10. The method in which the main control unit 10 controls the light emission of the light-emitting unit 60 can be called an in-vehicle light emission control method.

The in-vehicle device can be configured with hardware such as an integrated circuit, or a combination of hardware and software. Any specific function that is all or part of the functions realized by the in-vehicle device can be written as a program and the program can be stored in a memory that can be mounted on the in-vehicle device. The specific function can then be realized by executing the program on a program execution device (for example, a microcomputer that can be mounted on the in-vehicle device, such as the CPU of the main control unit 10). The program can be stored on any recording medium. The recording medium that stores the program can be mounted on or connected to a device different from the in-vehicle device.

The embodiments of the present invention can be modified in various ways as appropriate within the scope of the technical ideas set forth in the claims. The above embodiments are merely examples of the present invention, and the meanings of the terms of the present invention or each of the constituent elements are not limited to those described in the above embodiments. The specific numerical values shown in the above description are merely examples, and can, of course, be changed to various numerical values.

SYS In-vehicle system 1 Multimedia device 1
2 Camera section 2R Rear camera 3 GPS processing section 4 Vehicle-mounted sensor section 4A Shift lever sensor 4B Vehicle speed sensor 10 Main control section 11 Navigation section 12 Hands-free compatible section (HF compatible section)
13 Voice recognition support unit (VR support unit)
REFERENCE SIGNS LIST 14 Light emission control section 20 Memory section 30 Display screen 40 Operation section 41 Audio knob 50 Microphone 60 Light emission section 70 Communication module 80 Audio device 81 Tuner section 82 Audio amplifier 83 Speaker TM Terminal device NET Communication network CC1 Steering wheel CC2 Shift lever CC3 Ignition switch

Claims (10)

an audio knob for adjusting an output volume of an audio device installed in a vehicle;
a light emitting unit that illuminates the audio knob by emitting light;
a main control unit that controls execution of an audio function operation for reproducing an output sound signal of a tuner unit of the audio device and another function operation different from the audio function operation;
An in-vehicle device, wherein the main control unit causes the light-emitting unit to emit light in accordance with a first control when the audio function operation is executed, and causes the light-emitting unit to emit light in accordance with a second control different from the first control when the other function operation is executed. Further comprising a display screen for displaying an image;
2. The in-vehicle device according to claim 1, wherein the main control unit, when the other functional operation is executed, causes a predetermined image corresponding to the other functional operation to be displayed on the display screen, and when the display image on the display screen is changed from the predetermined image during the execution period of the other functional operation, causes the light-emitting unit to continue emitting light in accordance with the second control. 3. The in-vehicle device of claim 2, wherein when the execution period of the other functional operation includes a first period during which the specified image is displayed on the display screen and a second period after the display image on the display screen is changed from the specified image, and when the state of the other functional operation is the same between the first period and the second period, the main control unit causes the light-emitting unit to emit light in the same light-emitting mode in the first period and the second period. The other functional operation state is a plurality of states,
The in-vehicle device according to claim 3 , wherein the main control unit controls a light emission state of the light emitting unit in accordance with a current state of the other functional operation during an execution period of the other functional operation. 5. The in-vehicle device according to claim 2, wherein the main control unit changes the display image on the display screen from the predetermined image when a display change condition is satisfied during the execution period of the other functional operation. 6. The in-vehicle device according to claim 5, wherein when the main control unit receives an instruction from a user of the in-vehicle device to change the display image on the display screen from the specified image to a navigation image including a map image while the specified image is being displayed on the display screen during the execution period of the other functional operation, the display change condition is met and the main control unit changes the display image on the display screen from the specified image to the navigation image. 6. The in-vehicle device according to claim 5, wherein when the vehicle's traveling direction is set to a reverse direction while the specified image is being displayed on the display screen during the execution period of the other functional operation, the main control unit changes the display image on the display screen from the specified image to a camera image taken of the outside of the vehicle when the display change condition is met. 8. The in-vehicle device according to claim 1, wherein the other functions include a hands-free function that enables a user of the in-vehicle device to make a hands-free call. 9. The in-vehicle device according to claim 1, wherein the other functional operations include a voice recognition functional operation for recognizing speech content of a user of the in-vehicle device and responding to the speech content. 1. An in-vehicle light emission control method for use in an in-vehicle device including an audio knob for adjusting an output volume of an audio device installed in a vehicle and a light emitting unit provided in correspondence with the audio knob, comprising:
a main control step for controlling execution of an audio function operation for reproducing an output sound signal of a tuner unit of the audio device and another function operation different from the audio function operation;
In the main control step, when the audio function operation is executed, the light-emitting unit is caused to emit light in accordance with a first control, and when the other function operation is executed, the light-emitting unit is caused to emit light in accordance with a second control different from the first control.

Images