JP6417686B2 - Audio output sequence control program, audio output sequence control method, and audio output sequence control device - Google Patents

Description

  The present invention relates to a technique for controlling the audio output order of an object output by a computer.

In order to improve the convenience of the visually impaired and the like, the contents included in an object (for example, a document or a screen) output by a computer are read out and output as audio.
Here, when the content of the target object for voice output is a sentence, the sentence may be read as it is. However, the audio output object is not necessarily a sentence. For example, a plurality of output items such as a form and a slip display screen are arranged, and output data by the application is stored in each corresponding output item. In some cases, the object is output in the state of being touched. In the case of such an object, even if each output item is simply read out, it may be difficult to understand the contents.

  Thus, for example, the following techniques have been proposed as techniques for generating data to be read out in audio output. That is, in this technique, a unique document represented by a data structure unique to each application program is converted into a common document represented by a common data structure. Then, the data structure of the common document is converted so that the visually impaired can easily understand the contents when the voice is output by the text reader. Specifically, attributes such as position, size, color, font type and the like to be displayed by the corresponding application program for each of a plurality of objects included in the common document are determined in advance by the structure conversion unit. Whether or not a condition defined in accordance with a predetermined data structure such as a table structure is satisfied is detected. Then, the structure conversion unit converts the data structure of the common document by assigning each of the plurality of objects to each node of the default data structure that satisfies the condition.

JP 2007-249754 A

Here, in order to make it easier to understand the content of the audio output, it is desirable to output the audio in an order suitable for the semantic content of each output item. And in order to realize such audio output, the meaning content of the output item is more accurately determined than the information indicating the display mode such as position, size, color, font type, etc. as in the prior art described above. It is desirable to determine the audio output order of each output item using information that can be obtained. In addition, there is a method of manually defining the sound output order so that sound is output in an order suitable for the meaning content of each output item, but there are problems in the burden on the operator, maintainability, and the like.
Therefore, an object of one aspect of the present invention is to automatically determine an audio output order corresponding to attributes of output items included in data output by a computer.

In one aspect of the present invention, an attribute corresponding to each of a plurality of values included in data is specified from definition information of output data including a plurality of values as output items. Further, the attribute is associated with the output order according to the semantic classification in the sentence structure determined according to the attribute, and the semantic classification is associated with the condition to be satisfied by the output item corresponding to the semantic classification. The output order of the plurality of values is determined according to the attribute corresponding to each of the plurality of values with reference to the storage unit storing the corresponding relationship information. Then, referring to the sentence correction information indicating the sentence including the item and the character string to be complemented according to the item, and generating the corrected sentence including the output item based on the sentence correction information, The included voice definition information is generated, the information of each item of the output data is applied to the voice definition information, the voice output data is generated, and the output order when the voice output data is read and output as voice is controlled. The above determination process refers to the correspondence information, and when a specific character string is placed immediately before or immediately after the output item as the condition, the value that the semantic classification is output in the output item is It is determined that the semantic classification belongs, and the output order corresponding to the semantic classification is determined. Furthermore, the process of generating the speech definition information refers to information on a group composed of a plurality of item names and synonyms belonging to the group, and is included in the corrected sentence of the sentence correction information with other synonyms belonging to the same group. If there is an item that matches the item name, an amended sentence including the item name that coincides with the item name of the sentence correction information is generated.

  According to one aspect of the present invention, it is possible to automatically determine an audio output order corresponding to an attribute of an output item included in data output by a computer.

It is explanatory drawing which shows an example of the form data output in this embodiment. It is explanatory drawing which shows an example of the production | generation procedure of the audio | voice output data in the past. It is explanatory drawing which shows an example of the whole structure of the system in this embodiment, and a function structure of each server. It is explanatory drawing which shows an example of the form definition information (whole information) in this embodiment. It is explanatory drawing which shows an example of the form definition information (item detailed information) in this embodiment. It is explanatory drawing which shows an example of the output data by the business application in this embodiment. It is explanatory drawing which shows an example of the item extraction data according to form in this embodiment. It is explanatory drawing which shows an example of the item meaning dictionary in this embodiment. It is explanatory drawing which shows an example of the text correction dictionary in this embodiment. It is explanatory drawing which shows an example of the synonym dictionary in this embodiment. It is explanatory drawing which shows an example of the audio | voice definition information in this embodiment. It is explanatory drawing which shows an example of the audio | voice output data in this embodiment. It is explanatory drawing which shows an example of the form data with a voice code in this embodiment. It is a flowchart which shows an example of the production | generation process of the audio | voice definition information in this embodiment. It is a flowchart which shows an example of the production | generation process of the audio | voice output data generation in this embodiment, and the production | generation of form data with an audio code. It is a flowchart which shows an example of the dictionary update process in this embodiment, (A) shows an example of the update process of an item meaning dictionary, (B) shows an example of the update process of a text correction dictionary. It is explanatory drawing which shows an example of the form data output in this embodiment. It is a flowchart which shows an example of a part of audio | voice output data generation process in this embodiment. It is an example of the hardware constitutions of the form definition production | generation server in this embodiment, a form output terminal, and an audio | voice output data production | generation server.

[Background and Outline of this Embodiment]
In recent years, in order to improve convenience for visually handicapped persons and the like, the contents included in documents and screens are read out and output as voice. Specifically, there is a technique in which a voice code attached to a document is read by a voice output terminal (text reader) and the contents of the document are read out by voice.

  Here, when the object to be read out by the voice output terminal is a sentence such as a business letter, for example, the listener can understand it by reading it out as it is written. However, when the object of audio output is, for example, a form or the like, it is basically a list of items (for example, words, dates, numbers, etc.) instead of sentences. For this reason, even if the content of an item is simply read out, it may be difficult for the listener to understand the content.

  For example, in the case of the specific example of the form shown in FIG. 1, when simply reading from the top, “Invoice, Oct. 1, 2013, slip No. 123456, postal code 111-1111, Kamioda, Nakahara-ku, Kawasaki-shi, Kanagawa-ken XX-X, ABC Corporation, XYZ Corporation, postal code 999-9999, Chiyoda-ku, Tokyo (hereinafter abbreviated) ". When the user hears only such reading contents in a situation where the user cannot visually recognize the form, it is difficult to easily understand the contents as a whole. For example, in the case of such a delivery note, it is desirable to first know from whom to whom it was issued. In the above example, the address is read out before the name of the issuer. For this reason, the timing of reading out the name of the issue destination and the name of the issue source is delayed. Further, when the address of the issue destination is read out, it is not clear who the address is being read out. In this way, it is difficult to understand the entire contents simply by reading a form.

  In order to solve such a problem, when the object of audio output is a form or the like, voice definition information that specifies a reading range and a reading order included in the form is separately generated for each type of form. Yes. Then, voice output data (read-out data) indicating the read-out contents of the form is generated by matching the voice definition information with the form to be read out.

FIG. 2 shows an example of a conventional procedure for generating audio output data. In the example of the generation procedure, first, the user uses the form tool 901 to generate form definition information 911 that defines items and layouts included in the form (S921). The form tool 901 reads output data 912 (for example, a CSV (Comma-Separated Values) file) output by the application, generates form data 913 based on the form definition information 911, and outputs it (S922).
In the generation procedure, the form 913 is read by the voice output data generation tool 902, and characters included in the form are identified by, for example, OCR (Optical Character Reader) or the like (S923). Further, the user uses the voice output data generation tool 902 to perform area definition specifying the reading range and the like based on the identification result of the characters included in the form, and manually generate the voice definition information 914 (S924). ). Then, the voice output data generation tool 902 generates voice output data of the form based on the identification result of the characters included in the form and the voice definition information 914. Further, the voice output data generation tool 902 generates a voice code in which the voice output data is encoded, and generates form data 915 with voice code in which the voice code is added to the form (S925).

  Here, for example, when the form layout or item content is changed, the voice definition information 914 must also be changed. However, when the voice definition information 914 is manually generated as described above, it is a burden on the user to change the voice definition information 914 each time the layout of the form, the contents of items, and the like are changed. Further, in the mode in which the voice definition information 914 is manually generated, individual voice definition information is required for each form. In particular, when the number of forms to be managed increases, the burden on the user further increases.

  In view of such a conventional problem, in this embodiment, voice definition information is automatically generated. In addition, as an example of a technique for automatically generating data suitable for audio output, there is a technique disclosed in Japanese Patent Application Laid-Open No. 2007-249754, but in this embodiment, an audio output that is easier to understand is provided. In order to achieve this, information that can more accurately determine the meaning of each item is used.

  Specifically, in the present embodiment, item attribute information including information indicating the meaning content of the output item of the form is added to the form definition information that is the definition information of the output data generated in advance when the form is output ( (Meta information) is included. The information included in the item attribute includes, for example, the item name of the output item, the data format (for example, date item, numerical item, text item, etc.), the arrangement position in the form layout, and the like. And in this embodiment, in the production | generation of audio | voice definition information, these information contained in form definition information is acquired. As a result, it is possible to acquire accurate information that is a criterion for determining the audio output order of a plurality of values output in each item.

  In the present embodiment, the correspondence information in which the audio output order corresponding to the item attribute is set in advance is referred to. More specifically, the semantic content associated with the item attribute (for example, a semantic classification indicating which of “when”, “who”, “who”, “what”, etc., in the sentence structure corresponds) The item meaning information in which the sound output order corresponding to the information) is preset is referred to. Then, by collating the item attributes extracted from the form definition information with the item semantic information, the audio output order of the output items is automatically determined. Then, voice definition information including information indicating the voice output order is automatically generated, and voice output data is generated based on the voice definition information.

  Thus, in this embodiment, since the audio output order of the output items is automatically determined and the audio definition information is automatically generated, it is not necessary for the user to generate the audio definition information manually. Therefore, the user's work load is reduced, and it is possible to easily cope with changes in the form layout, item contents, etc., and the maintainability is improved. In this embodiment, when determining the audio output order of the output items, an item attribute including information indicating the meaning content of each output item is acquired from the form definition information. For this reason, it is possible to determine a high-accuracy audio output order suitable for the meaning content of the output item, and to read out the form data in the output order.

  Note that an example of an object configured by a set of output items is not limited to a form. For example, as another example of an object in which a plurality of output items are arranged and output data by an application is stored in each corresponding output item, such as a form, a layout of a WEB screen There is a WEB screen generated using the definition information (template). Still another example is a document output by DTP (Desktop Publishing) software. In the present embodiment, the audio output target is a form and will be described using an example in which item attributes are acquired from the form definition information. However, such other objects that use the same information as the form definition information are used. The technique described in this embodiment can also be applied when outputting audio.

[First embodiment]
<Overall system configuration and functional configuration of each server>
FIG. 3 shows an example of the overall configuration of the system and the functional configuration of each server in the present embodiment.
The system includes a form definition generation server 1, a form output terminal 2, and an audio output data generation server 3. The form definition generation server 1, the form output terminal 2, and the audio output data generation server 3 are connected to each other via a network so that they can communicate with each other. The network is, for example, a LAN (Local Area Network) or a WAN (Wide Area Network), and may be either a wired connection or a wireless connection.

  The form definition generation server 1 is a server in which, for example, a system administrator or the like generates a form definition using a form design tool or the like. As shown in FIG. 3, the form definition server includes a form definition generation unit 11 (for example, a form design tool) that generates a form definition. Further, the form definition generation server 1 holds the form definition information 12 generated by the form definition generation unit 11 in the storage unit.

4 and 5 show an example of the form definition information 12 corresponding to the specific example of the form shown in FIG. FIG. 4 shows overall information indicating the layout of the entire form of the form definition information 12, and FIG. 5 shows detailed information of each item included in the form.
For example, as shown in FIG. 4, the overall information includes a form name indicating a form type, a partition name (header part, body part or footer part) in the form, an arrangement form of the partition (whether the position is fixed or floating), It contains information on the start position and vertical width of the partition. The overall information includes details of each partition, that is, information indicating items included in each partition.

  The item detailed information is, for example, a form name, an item name of each item included in the form, an item type (information specifying whether it is a number or text), an item length, and item arrangement information, as shown in FIG. Contains information such as vertical position and horizontal position. Furthermore, the item detailed information includes further detailed information regarding the item, if necessary. As an example of detailed information, classification of items (detailed information of item type, for example, in the case of numbers, indicates whether it is a simple numerical value, date, currency, or the like). In this specification, information including both the item type and the classification, that is, information that can specify a text item, a date item, a numerical item, and the like is referred to as an item data format. Further, the detailed information includes information of an edit format character string (for example, in the case of a date, written as XX year XX month XX day, etc.) as necessary.

  The form output terminal 2 is a server that outputs data output from individual business applications based on the form definition information 12. As shown in FIG. 3, the form output terminal 2 includes a business application 21 used by a user (operator or the like), and a form output unit 22 that outputs data (for example, CSV data) output by the business application 21. The form output terminal 2 holds the output data 23 from the business application 21 and the form data 24 of the output data 23 output by the form output unit 22 in the storage unit.

  FIG. 6 is an example of output data 23 (CSV data) from the business application 21. The specific data arrangement order of each item output in the output data 23 corresponds to the item arrangement order defined in the form definition shown in FIGS. Thereby, the form output unit 22 can generate the form data 24 using the form definition information 12, while the audio output data generation unit 34 of the audio output data generation server 3 uses the audio definition information 45 described later. Audio output data 46 can be generated. An example of the form data 24 (an example of the form on which the form data 24 is printed) is the same as that shown in FIG.

The audio output data generation server 3 is a server that performs various processes so that the form output by the form output terminal 2 can be read aloud.
As shown in FIG. 3, the voice output data generation server 3 realizes its function by executing a program. The item information extraction unit 31, the voice output order determination unit 32, the sentence correction unit 33, the voice output data A generation unit 34, a voice code generation / synthesis unit 35, and a dictionary update unit 36 are provided. In addition, the voice output data generation server 3 stores the item-specific item extraction data 41, the item meaning dictionary 42, the sentence correction dictionary 43, the synonym dictionary 44, the voice definition information 45, the voice output data 46, the voice code-added form data in the storage means. 47.

  The item information extraction unit 31 acquires the form definition information 12 generated by the form definition generation server 1 and reads the entire information and item detailed information of the form definition information 12. The item information extraction unit 31 includes item attributes including information indicating the semantic contents of each item in the form type indicated by the form name of the form definition information 12, for example, item names, data formats, and items. The arrangement position indicating the existing partition is extracted (specified), and the item-specific item extraction data 41 is generated. The item information extraction unit 31 is an example of a specifying unit.

  The voice output order determination unit 32 refers to the item meaning dictionary 42 in which the voice output order corresponding to the meaning contents of the item is set, and based on the meaning contents indicated by the item meaning dictionary 42 and the item attribute of each item, Determine the audio output order of items. Specifically, for example, the voice output order determination unit 32 collates the item attribute of each item with the item attribute associated with the semantic classification in the sentence structure in the item semantic dictionary 42, and the semantic classification of each item. Is determined. And the audio | voice output order determination part 32 determines the audio | voice output order of each item based on the said semantic classification. The audio output order determination unit 32 is an example of a determination unit.

  The sentence correction unit 33 refers to the sentence correction dictionary 43 indicating the item and the sentence including the character string to be complemented according to the item, and becomes a sentence suitable for voice output for each item for which the voice output order is determined. In this way, corrections are made to supplement the wording. The sentence correction unit 33 displays a user interface, for example, and accepts manual sentence correction by the user. Then, the sentence correction unit 33 outputs the speech definition information 45 including the sentence generated as a result of the correction.

  The audio output data generation unit 34 acquires the output data 23 from the business application 21 from the form output terminal 2, applies the information of each item of the output data 23 to the audio definition information 45, and generates the audio output data 46. In other words, the process of generating the audio output data 46 reads a plurality of values of the output data 23 (that is, a plurality of values output in the form data 24) based on the determined audio output order. This controls the order of voice output. The audio output data generation unit 34 is an example of a control unit.

The voice code generation / synthesis unit 35 generates a voice code obtained by converting the voice output data 46 into a code readable by the voice output terminal. Then, the voice code generating / synthesizing unit 35 acquires the form data 24 from the form output terminal 2, and generates form data 47 with voice code obtained by synthesizing the voice code with the form data 24.
The dictionary update unit 36 receives an instruction to update the item meaning dictionary 42 by a user (system administrator or the like), for example, through a user interface, and updates the item meaning dictionary 42. Similarly, the dictionary update unit 36 receives an instruction to update the text correction dictionary 43 from the user, and updates the text correction dictionary 43.

  The item-specific item extraction data 41 is data for storing information on extracted items for each type of form. For example, as illustrated in FIG. 7, the item extraction data 41 for each form includes, for each form type indicated by the form name, the [item name] of each item included in the form, the [data format] of the item, and the item. [Disposition position] information indicating the partitions that are arranged is included. The [arrangement position] may be information indicating specific coordinates where each item is arranged, for example.

  The item meaning dictionary 42 is a table in which the correspondence between item attributes and sound output order is defined. Specifically, in the item meaning dictionary 42, an item attribute indicating the corresponding semantic content is associated with the semantic classification of the item in the sentence structure, which is determined based on the semantic content of the item, and an output corresponding to the semantic classification is provided. It is a table in which the order is associated. The item meaning dictionary 42 is an example of correspondence information. For example, as shown in FIG. 8, the item meaning dictionary 42 includes [output order] indicating the output order of items in the sentence structure, [semantic category] corresponding to the item attribute, and a condition corresponding to each semantic category. [Data format], [Item name] attached to the item, [Arrangement position] indicating the partition where the item is located, and individual conditions that are used as other judgment materials for classifying the item Includes [Remarks] information. The order of each semantic classification in the table (the order of “when”, “where”, “who”, “who”, “what”, “how”) is the sound of these semantic categories. The output order is shown. The item meaning dictionary 42 may explicitly hold data indicating the sound output order.

  When the sentence correction dictionary 43 generates the audio output data 46 based on each item, the sentence correction dictionary 43 corrects the sentence according to the meaning content of each item so that it becomes a meaningful document instead of a simple list of items. This is a table storing text information (character strings). For example, as shown in FIG. 9, the sentence correction dictionary 43 shows [corrected sentence] indicating a corrected sentence including an item and a character string to be complemented according to the item, a format to be converted when fitting to the sentence, etc. [Remarks] information is displayed. In [corrected text], an item included in the corrected text is expressed by an individual item name (for example, “total amount”) or an item data format (for example, “date item”). When it is expressed in the data format, the entire items corresponding to the data format are applicable.

  The synonym dictionary 44 absorbs the fluctuation of the item name and improves the matching accuracy with each dictionary when classifying each item based on the item meaning dictionary 42 or correcting the item based on the sentence correction dictionary 43. This is a table in which synonyms are defined. For example, as shown in FIG. 10, the synonym dictionary 44 includes information on [group] indicating a group of a plurality of item names having the same meaning and [synonym] belonging to the group.

  The audio definition information 45 includes data indicating the audio output items included in the form and the audio output order for each type of form, and is data indicating how to read each item. For example, as shown in FIG. 11, the audio definition information 45 specifically indicates how to output audio based on semantic classification and the like in order from the top. In the voice definition information 45 shown in FIG. 11, information on individual output data 23 (CSV data) corresponding to the item is stored for a part where an item name or the like is indicated by {}, such as {issuer name}. To do.

The voice output data 46 is data generated by storing the information of the individual output data 23 in the voice definition information 45. For example, as shown in FIG. 12, the voice output data 46 is text data read directly in voice output.
The form data 47 with sound code is data obtained by combining the form data 24 with a sound code obtained by converting the sound output data 46 into a format readable by the sound output terminal. FIG. 13 shows an example of printed form data 47 with voice code. As shown in FIG. 13, the voice code-added form data 47 has a voice code attached to a corner (for example, upper left) of the form. When the audio code is read by an audio output terminal used by a visually impaired person or the like, the audio output terminal can output (read out) the content of the audio output data 46.

  Note that the system configuration shown in FIG. 3 is merely an example, and for example, the functions of the form definition generation server 1, the form output terminal 2, and the audio output data generation server 3 may be realized by a single computer. Further, for example, the voice output data generation server 3 may share the form definition information 12 with the form definition generation server 1 or the form data 24 with the form output terminal 2 by using a distributed shared memory or the like. Further, the audio output data generation server 3 may acquire such information by a storage medium such as a CD-ROM.

<Description of processing>
The voice definition information generation process executed by the item information extraction unit 31, the voice output order determination unit 32, and the sentence correction unit 33 of the voice output data generation server 3 will be described with reference to the flowchart shown in FIG. In the description of the process, a specific example of generating the voice definition information 45 in the “delivery form” form shown in FIG. In this specific example, the description will be made assuming that the voice definition information 45 of the “delivery note” and the item-specific item extraction data 41 do not exist in the storage means at the start of processing.

In step 1, the item information extraction unit 31 reads the form definition information 12 generated by the form definition generation server 1.
(In the example of the delivery note in FIG. 1, the form definition information 12 shown in FIGS. 4 and 5 is read.)
In step 2, the item information extraction unit 31 acquires a form name from the form definition information 12.
(In the example of the delivery note in FIG. 1, the “delivery note”, which is the form name, is acquired.)

In step 3, the item information extraction unit 31 determines whether or not the voice definition information 45 corresponding to the read form name already exists in the storage means, that is, the voice type definition information 45 of the form type indicated by the form name has already been obtained. It is determined whether or not it has been generated. If the voice definition information 45 exists, the process proceeds to step 4 (Yes), and if not, the process proceeds to step 5 (No).
In step 4, the voice output data generation unit 34 acquires voice definition information 45 corresponding to the form name from the storage unit.

In step 5, the item information extraction unit 31 determines whether or not the item-specific item extraction data 41 corresponding to the form name exists. If the form-specific item extraction data 41 exists, the process proceeds to step 6 (Yes), and if not, the process proceeds to step 7 (No).
In step 6, the item information extraction unit 31 selects and acquires item-specific item extraction data 41 corresponding to the read form name.

In step 7, the item information extraction unit 31 extracts item attributes (item name, data format, and arrangement position in this embodiment) of each item included in the form definition information 12 from the form definition information 12 read out in step 1. To do. Then, for each form type of the form name, form-specific item extraction data 41 storing the item attributes of each extracted item is generated.
(In the example of the delivery note in FIG. 1, the arrangement position indicating the partition where each item is arranged (for example, “header part” in the case of the “issue date” item) from the entire information shown in FIG. Further, from the item detailed information shown in Fig. 5, the item name of each item (for example, "issue date" for "issue date") and the data format (eg, "issue date"). (The date is identified from the item type and classification of the item detail information), and the item-specific item extraction data 41 shown in FIG.

  In step 8, the audio output order determination unit 32 refers to the item meaning dictionary 42 and compares it with the item attributes of each extracted item to determine which semantic category each item belongs to. Specifically, for each item, the generation order generation unit corresponds to the [data format] information in the item semantic dictionary 42 and the item name corresponds to the [item name] information in the item semantic dictionary 42. The meaning classification that corresponds to the information of [arrangement position] of the item meaning dictionary 42 and the item satisfies the condition of the information of [remarks] of the item meaning dictionary 42 It is determined as the semantic classification of the item. The voice output order determination unit 32 refers to the synonym dictionary 44 in the matching process. For example, even if the item name does not match any [category name] of any semantic classification of the item semantic dictionary 42, the [item name] of any semantic classification of the item semantic dictionary 42 is another synonym belonging to the same group. If there is a match, the processing is performed assuming that it matches the [item name] of the semantic classification. For items that do not fall under any semantic classification, the semantic classification is undecided.

(In the example of the delivery note in FIG. 1, for example, the item “issue date” has an item name, a data format, and an arrangement position of “when” in the item meaning dictionary 42 shown in FIG. Since it corresponds to [item name], [data format] and [arrangement position] of the data of, it is determined that the semantic classification is “when.” Similarly, for example, “customer name” is an item name, data Each of the format and the arrangement position corresponds to [item name], [data format] and [arrangement position] of the data whose semantic classification in the item semantic dictionary 42 shown in FIG. 8 is “to whom”, and FIG. According to the overall information of the form definition shown in the above, since “sama” (fixed character string) is immediately after “customer name”, it also meets the requirements of [Remarks] for data whose semantic classification is “who” Therefore, it is determined that the semantic classification is “who”. To have been placed "issuer address", such as "issuer phone number" does not correspond to any of the semantic classification. Because of this, meaning classification of these items shall be undecided.)

  In step 9, the audio output order determination unit 32 refers to the item meaning dictionary 42 and determines the audio output order of items based on the result of classifying each item in step 9. Specifically, the voice output order determination unit 32 determines the voice output order of the items based on the output order defined in the item semantic dictionary 42 as “when”, “where”, “who”, “who”, The order of semantic classification of “what” and “what” is used. For example, among items whose semantic classification is undetermined, items included in a detailed part of a form (a part in which the same item is repeatedly displayed according to output data and includes a plurality of the same item) May be output after all items whose semantic classification is determined. In the present embodiment, it is possible to specify the detailed portion by the arrangement position being the “body portion”. Further, for example, when information indicating individual coordinates is extracted as the arrangement position, it can be specified based on whether or not the coordinate position corresponds to the detailed portion.

(In the example of the delivery note in FIG. 1, “issue date” corresponding to “when”, “issuer” corresponding to “who”, “who” as a result of matching with the item meaning dictionary 42 shown in FIG. “Customer name” corresponding to “N”, “Voucher number” corresponding to “What”, and “Total amount” corresponding to “What”. Of the items that have not been determined, the “product name”, “money amount”, and “quantity” arranged in the body part are output as audio after each item for which the semantic classification is determined.

  In step 10, the sentence correction unit 33 performs correction so that the audio output content of each item for which the audio output order has been determined in step 10 is not a simple list of items but a meaningful sentence. Specifically, the sentence correction unit 33 refers to the sentence correction dictionary 43, extracts a corrected sentence including each item, and generates a sentence including the item. Note that the sentence correction unit 33 also refers to the synonym dictionary 44 when extracting the corrected sentence including each item from the sentence correction dictionary 43 in this way, and uses other synonyms belonging to the same group in the sentence correction dictionary 43. If there is an item that matches the item name included in the corrected text, processing is performed assuming that the item name matches the [item name] in the text correction dictionary 43.

(In the example of the delivery note in FIG. 1, for example, “issue date” is a date item. Therefore, based on the text correction dictionary 43 shown in FIG. 9, “issue date is {issue year} year { "Issuance month} month {issuance date}", and "issuer name" is "this {form name} is {address name} {address name} from {issuer name} in the text correction dictionary 43. }, “Customer name” belongs to the same group as “destination name” in the synonym dictionary 44 shown in FIG. For example, it can be corrected that “this delivery note has been issued to {customer name} by {issuer name}”.)

In step 11, the sentence correction unit 33 displays a user interface, for example, as necessary, and accepts manual sentence correction by the user.
(In the example of the delivery note in FIG. 1, for example, at the start of voice output by a manual correction instruction, a description at the beginning such as “This is {form name}” is inserted. Add a fixed phrase such as “It ’s over”.)

In step 12, the sentence correction unit 33 outputs the voice definition information 45 of the form type indicated by the form name based on the voice output order determined in step 9 and the sentence that is the result corrected in step 10.
(In the example of the delivery note in FIG. 1, the voice definition information 45 shown in FIG. 11 is output. Note that the sentence example and the like added in step 11 described above are routinely added in the generation of the voice definition information 45. May be.)

Next, the flowchart shown in FIG. 15 is shown for the process of generating the voice output data and the form data with the voice code executed by the voice output data generating unit 34 and the voice code generating / synthesizing unit 35 of the voice output data generating server 3. The description will be given with reference.
In step 21, the audio output data generation unit 34 acquires the output data 23 output from the form output terminal 2.
(In the example of the delivery note in FIG. 1, the output data 23 shown in FIG. 6 is acquired.)

  In step 22, the audio output data generation unit 34 reads the acquired output data 23 and audio definition information 45, and generates audio output data 46 using the output data 23 and audio definition information 45. Specifically, the voice output data generation unit 34 substitutes individual values included in the output data 23 of the item for items included in the voice definition information 45 to complete a sentence to be output as voice. The text data indicating the sentence is referred to as voice output data 46.

(In the example of the delivery note in FIG. 1, the voice output data 46 is as shown in FIG. 12. For example, “this delivery note is {customer name} from {issuer name} in the voice definition information 45 shown in FIG. In the case of the sentence “issued by name}”, “XYZ Corporation” corresponding to “issuer name” and “ABC shares” corresponding to “customer name” in the output data 23 shown in FIG. “Company” is substituted, and the voice output data is “This delivery note has been issued to ABC Corporation by XYZ Corporation.”

In step 23, the voice code generation / synthesis unit 35 generates a voice code obtained by converting the voice output data 46 into a code readable by the voice output terminal.
In step 24, the voice code generation / synthesis unit 35 acquires the form data 24 output from the form output terminal 2, and synthesizes the voice code with the form data 24 to generate form data 47 with voice code.
(In the example of the delivery note in FIG. 1, as shown in FIG. 13, form data 47 with voice code in which a voice code is added to the corner of the delivery note in FIG. 1 is generated.)

Next, the item meaning dictionary update processing executed by the dictionary update unit 36 of the voice output data generation server 3 will be described with reference to the flowchart shown in FIG.
In step S31, the dictionary update unit 36 displays a user interface, for example, and accepts an instruction to update the item meaning dictionary 42 by the user.
In step S32, the dictionary update unit 36 updates the item meaning dictionary 42 based on the update instruction.

Next, the sentence correction dictionary update processing executed by the dictionary update unit 36 of the voice output data generation server 3 will be described with reference to the flowchart shown in FIG.
In step S41, the dictionary update unit 36 displays a user interface, for example, and accepts an instruction to update the sentence correction dictionary 43 by the user.
In step S32, the dictionary update unit 36 updates the sentence correction dictionary 43 based on the update instruction.

<Effects of this embodiment>
According to the present embodiment, since the audio definition data 45 is automatically generated in the audio output data generation server 3, it is not necessary for the user to generate the audio definition information 45 manually. Therefore, the user's work load is reduced, and it is possible to easily cope with changes in the form layout, item contents, etc., and the maintainability is improved. In the present embodiment, the item attributes of each item are acquired from the form definition information 12 as well as merely automatically generated. This item attribute includes information indicating the meaning content of the item, such as an item name and a data format. Since the audio output order of the items is determined based on the item attributes, audio output suitable for the semantic content of the items can be realized.

  Specifically, in this embodiment, the item classification acquired from the form definition information 12 is compared with the item meaning dictionary 42 to determine the semantic classification. At this time, the item attribute includes information indicating the semantic content, and the item meaning dictionary 42 associates the semantic classification according to the item attribute, so that the semantic classification can be appropriately determined. . In the item meaning dictionary 42, since the voice output order corresponding to the meaning content is defined, the voice output order of each item is appropriately determined. For this reason, in the present embodiment, it is possible to perform highly accurate voice output that matches the semantic content of each item of the form.

Also, according to the present embodiment, each item is corrected based on the sentence correction dictionary 43 so that it becomes a meaningful document rather than a simple list of items. For this reason, a voice output that is easy to understand for the user who listens to the contents of the voice output is made.
Further, according to the present embodiment, items whose semantic classification is not determined are excluded from the targets of audio output in principle. In other words, items that should not be subject to audio output can be excluded from audio output targets by setting the item meaning dictionary 42 so as not to correspond to any semantic classification. For this reason, for example, by outputting audio, it becomes easy to miss other important information, and information that may interfere with the understanding of the entire content (for example, an address that becomes longer when audio is output, etc. Can be excluded from the audio output targets. Therefore, it is possible to realize an audio output that is easier for the user to understand.

  Further, according to the present embodiment, the items included in the specification part are output as audio after other items. As a result, for example, in the specific example of the delivery note described above, the “total amount” is output first by voice, and the detail portion is output by voice later. For this reason, the user who listens to the audio output can hear the items that are fixedly output first, regardless of the number of detail lines, and can easily grasp the entire contents.

  Further, according to this embodiment, the user can update the item meaning dictionary 42 as appropriate. For this reason, the semantic classification can be appropriately determined according to the situation. Similarly, the user can update the sentence correction dictionary 43 as appropriate. For this reason, it can adjust so that each item may be corrected into an appropriate sentence according to a situation.

  Further, according to the present embodiment, when the semantic classification is determined, by storing detailed conditions (arrangement of the preceding and following character strings, etc.) in the remarks of the item semantic dictionary 42, a highly accurate semantic classification can be obtained. A determination can be made. Note that the content of the remarks in the item meaning dictionary 42 in FIG. 9 is merely an example.

  In addition, according to the present embodiment, item information once extracted from the form definition is stored in the form-specific item extraction data 41. For this reason, when the voice definition information 45 of the same form type is generated next time, the form-specific item extraction data 41 can be used again. Similarly, according to the present embodiment, the form type for which the voice definition information 45 has been generated can be used again without generating the voice definition information 45 again. For this reason, the process performed for the production | generation of audio | voice output data can be abbreviate | omitted suitably.

  In the above embodiment, the item meaning dictionary 42 has been described as an example of a data configuration that does not depend on the form type. However, a different item meaning dictionary may be used for each form type. By doing so, collation accuracy with the item meaning dictionary in the items specific to each form type is increased, and the semantic classification can be determined more appropriately. Also, a different sentence correction dictionary may be used for each form type. By doing so, it is possible to perform sentence correction reflecting contents specific to the form type, and it is possible to realize more appropriate voice output.

  In addition, as described above, the technique described in the present embodiment is configured such that a plurality of output items are arranged regardless of the form, and output data by the application is output in a state where each output item is stored. Such an object can be applied to other objects.

[Second Embodiment]
In the second embodiment, in addition to the processing performed in the first embodiment, when the audio output data 46 is generated by the audio output data generation unit 34, the audio output data 46 is generated according to the breakdown of the data of the detailed portion to be output. A process of omitting part of the content is further performed. Note that the description of the same contents as in the first embodiment is omitted in principle.

  FIG. 17 shows an example of the form data 24 output in the second embodiment (an example of the form on which the form data 24 is printed). In the example of the form data 24, details with a quantity and an amount of 0 are consecutive with “refrigerator”, “electric washing machine”, and “DVD player”. When the detailed part of such a form is output by voice according to the first embodiment, “3 color TVs are 360000 yen, 1 karaoke set is 82000 yen, 0 refrigerators are 0 yen, electric laundry The price is 0 yen for 0 machines, 0 yen for DVD players, 23000 yen for 2 vacuum cleaners, and 70000 yen for 7 desk lamps. "

  In this way, when all the details with the quantity or amount of money being 0 are output by voice, there is a high risk that the details with a quantity or amount of money other than 0 that should be heard more accurately will be missed. For this reason, in the second embodiment, the details whose quantity or amount is 0 are collectively output as voice.

<Description of processing>
Of the audio output data generation processing executed by the audio output data generation unit 34 in the second embodiment, the detailed processing of the output data 23 output by the form output terminal 2 will be described with reference to the flowchart shown in FIG. explain. This description explains in detail a part of the processing in the second embodiment among the audio output data generation processing in step S22 shown in FIG.

In step S51, the audio output data generation unit 34 resets the counter to zero.
In step S <b> 52, the audio output data generation unit 34 reads out one line of data in the detailed portion of the output data 23. If there is no data to be read, the process is terminated as it is.

In step S53, the audio output data generation unit 34 determines whether or not the value of the item corresponding to the quantity or the like (for example, the quantity or the amount) of the read details is 0 or NULL. If it is 0 or NULL, the process proceeds to step S54 (Yes), and if not, the process proceeds to step S59 (No).
In step S54, the audio output data generation unit 34 accumulates the data of the item name item in the read specification, and adds 1 to the counter value.

In step S55, the audio output data generation unit 34 prefetches the next data of the detail portion by one line. If there is no prefetched data, the processing after step S57 is performed.
In step S56, the audio output data generation unit 34 determines whether the value of the item corresponding to the quantity of the details prefetched in step S55 is 0 or NULL. If it is 0 or NULL, the process returns to step S54 (Yes), and if not, the process proceeds to step S57 (No).

In step S57, the audio output data generation unit 34 determines whether or not the value of the counter is greater than 1, the process proceeds to step S58 if it is greater than 1 (Yes), otherwise the process proceeds to step S59. Proceed (No).
In step S58, the audio output data generation unit 34 outputs the product name item data accumulated in step S54 as audio. Specifically, for example, “Product name 1, Product name 2,...

In addition, in the case of the delivery note example of FIG. 17 described above, when voice is output by the process of the second embodiment, “three color TVs are 360000 yen, one karaoke set is 82,000 yen, refrigerator, "There are 0 electric washing machines and DVD players for 0 yen, 2 electric vacuum cleaners for 23000 yen, 7 electric lamps for 70000 yen".
In step S59, the audio output data generation unit 34 generates audio output data as usual according to the audio definition information 45 for the details read in step S52.

<Effects of this embodiment>
According to the second embodiment, the following effects are obtained in addition to the effects of the first embodiment. In other words, in the second embodiment, since the details with the quantity and the amount of money are collectively output, the risk of missing the sound output of the details with the quantity and the amount other than 0 should be heard more accurately. Can be reduced.

[Hardware configuration, etc.]
An example of a hardware configuration of a computer functioning as the above-described form definition generation server 1, form output terminal 2, and audio output data generation server 3 is shown in FIG. The computer includes a processor 101, a memory 102, a storage 103, a portable storage medium drive device 104, an input / output device 105, and a communication interface 106.

  The processor 101 includes a control unit, an arithmetic unit, an instruction decoder, and the like. The execution unit follows the instructions of the program decoded by the instruction decoder, and performs arithmetic / logic using the arithmetic unit according to a control signal output from the control unit. Perform the operation. The processor 101 has a TLB that functions as a control register in which various information used for control is stored, a cache that can temporarily store the contents of the already accessed memory 2 and the like, and a page table cache of virtual memory. Prepare. The processor 101 may have a configuration in which a plurality of CPU (Central Processing Unit) cores are provided.

  The memory 102 is a storage device such as a RAM (Random Access Memory), for example, and is a main memory in which a program to be executed by the processor 101 is loaded and data used for processing of the processor 101 is stored. The storage 103 is a storage device such as an HDD (Hard Disk Drive) or a flash memory, and stores programs and various data. The portable storage medium driving device 104 is a device that reads data and programs stored in the portable storage medium 107. The portable storage medium 107 is, for example, a magnetic disk, an optical disk, a magneto-optical disk, or a flash memory. The processor 101 executes a program stored in the storage 103 or the portable storage medium 107 while cooperating with the memory 102 or the storage 103. Note that the program executed by the processor 101 and data to be accessed may be stored in another device that can communicate with the computer. Note that the storage means of the form definition generation server 1, the form output terminal 2, and the audio output data generation server 3 described in this embodiment are the memory 102, the storage 103, the portable storage medium 107, or the other that can communicate with the computer. At least one of the following devices.

The input / output device 105 is, for example, a keyboard, a touch panel, a display, or the like, and receives an operation command by a user operation or the like, and outputs a processing result by a computer.
The communication interface 106 can include, for example, a radio frequency receiver and transmitter, and an optical receiver and transmitter in addition to a LAN (Local Area Network) card, for example. The aforementioned receivers and transmitters can be implemented to operate with one or more communication networks, such as a Wi-Fi network, a Bluetooth network, and long term evolution.
Each component of these computers is connected by a bus 108.

<Others>
Note that the functional configuration and physical configuration of the computer described in this specification are not limited to the above-described aspects. For example, the functions and physical resources are integrated and implemented, or conversely, are further distributed. It is also possible to implement.
In addition, in this specification, the description places “to be more than” and “to be less than” in comparison with the threshold value and the like are not limited to the description except for special cases, It can be appropriately replaced with “less than” or “less than (less than)”.

Regarding the above embodiment, the following additional notes are disclosed.
(Appendix 1)
The attribute corresponding to each of a plurality of values included in the data is specified from the definition information of the output data including the plurality of values as output items,
With reference to the correspondence information between the attribute and the output order stored in the storage unit, the output order of the plurality of values is determined according to the attribute corresponding to each of the plurality of values,
Based on the determined output order, control the order in which the plurality of values are read out and output as audio,
An audio output order control program that causes a computer to execute processing.

(Appendix 2)
Further including a process of generating audio definition information including information indicating an output order of the output items,
The process of generating the voice definition information refers to sentence correction information indicating a sentence including an item and a character string to be complemented according to the item, and based on the sentence correction information, a corrected sentence including the output item And generating the speech definition information including the corrected text. The speech output order control program according to appendix 1, wherein:

(Appendix 3)
The correspondence relationship information is determined according to the attribute, and the semantic classification in the sentence structure is associated with the output order according to the semantic classification,
The determining process refers to the correspondence relationship information, specifies the semantic classification corresponding to each of the plurality of values output in the output item, and determines an output order according to the semantic classification. The audio output order control program according to appendix 1 or 2, characterized by the above.

(Appendix 4)
The correspondence relationship information is further associated with the semantic classification and a condition to be satisfied by an output item corresponding to the semantic classification,
The specifying process is such that when the output item satisfies the condition associated with the semantic category in the correspondence information, the semantic category is a semantic category to which a value output by the output item belongs. The audio output sequence control program according to appendix 3, characterized by discriminating.

(Appendix 5)
The audio output order control program according to appendix 4, wherein the condition includes a condition in which a specific character string is arranged immediately before or after the output item.

(Appendix 6)
The attribute includes information indicating whether or not the same item is included in the detailed part of the output data for the output item,
In the determining process, based on the attribute of the output item, the output order of values output in a plurality of output items included in the specification part is set after the output order of values output in other output items. The audio output order control program according to any one of appendices 1 to 5, characterized in that:

(Appendix 7)
The sound according to any one of appendices 1 to 6, wherein in the determining process, a part of the plurality of values is excluded from a target of sound output according to an attribute of the plurality of values. Output sequence control program.

(Appendix 8)
The voice output order control program according to any one of appendices 1 to 7, further comprising a process of accepting an instruction to update the correspondence information and updating the correspondence information based on the update instruction. .

(Appendix 9)
The voice output order control program according to appendix 2, further comprising a process of receiving an instruction to update the sentence correction information and updating the sentence correction information based on the update instruction.

(Appendix 10)
The control process includes a process of generating audio output data corresponding to the output data based on an output order of the plurality of values.
The processing for generating the audio output data omits a part of the audio output content of the detailed portion according to the data content of the detailed portion of the output data. The audio output order control program described in 1.

(Appendix 11)
The process of generating the audio output data omits a part of the audio output content of the data indicating the details in which the value of the item indicating the quantity or the amount is zero among the data contents of the detailed portion included in the output data The audio output order control program according to appendix 10, wherein

(Appendix 12)
The audio output order control program according to any one of appendices 1 to 11, wherein the data is data to be output as a form, and the definition information of the output data is form definition information.

(Appendix 13)
The audio output order control program according to any one of appendices 1 to 12, wherein the attribute includes an item name and a data format of the output item.

(Appendix 14)
The attribute corresponding to each of a plurality of values included in the data is specified from the definition information of the output data including the plurality of values as output items,
With reference to the correspondence information between the attribute and the output order stored in the storage unit, the output order of the plurality of values is determined according to the attribute corresponding to each of the plurality of values,
Based on the determined output order, control the order in which the plurality of values are read out and output as audio,
An audio output order control method, wherein a computer executes processing.

(Appendix 15)
An identification unit that identifies an attribute corresponding to each of a plurality of values included in the data from definition information of output data including the plurality of values as output items;
A determination unit that refers to correspondence information between the attribute and the output order stored in the storage unit, and determines an output order of the plurality of values according to an attribute corresponding to each of the plurality of values;
An audio output order control device comprising: a control unit that controls the order in which the plurality of values are read out and output as audio based on the determined output order.

DESCRIPTION OF SYMBOLS 1 ... Form definition generation server, 2 ... Form output terminal, 3 ... Audio | voice output data generation server, 11 ... Form definition generation part, 12 ... Form definition information, 21 ... Business application, 22 ... Form output part, 23 ... Output data, 24 ... form data, 31 ... item information extraction unit, 32 ... audio output order determination unit, 33 ... sentence correction unit, 34 ... audio output data generation unit, 35 ... audio code generation / synthesis unit, 36 ... dictionary update unit, 41 ... Item-specific item extraction data, 42 ... Item meaning dictionary, 43 ... Sentence correction dictionary, 44 ... Synonym dictionary, 45 ... Audio definition information, 46 ... Audio output data, 47 ... Form data with audio code

Claims (7)

The attribute corresponding to each of a plurality of values included in the data is specified from the definition information of the output data including the plurality of values as output items,
The attribute is associated with the output order according to the semantic classification in the sentence structure determined according to the attribute, and the semantic classification is associated with the condition to be satisfied by the output item corresponding to the semantic classification. With reference to the storage unit in which the corresponding relationship information is stored, the output order of the plurality of values is determined according to the attribute corresponding to each of the plurality of values,
By referring to the sentence correction information indicating the sentence including the item and the character string to be complemented according to the item, and generating the corrected sentence including the output item based on the sentence correction information, the corrected sentence is included. Voice definition information
Applying the information of each item of the output data to the audio definition information to generate audio output data, causing the computer to execute a process for controlling the output order when reading out the audio output data and outputting the audio,
The determining process refers to the correspondence information, and when the specific character string is arranged immediately before or immediately after the output item as the condition, the value that the semantic classification is output in the output item. Is determined to belong to the semantic classification, the output order is determined according to the semantic classification,
The process of generating the voice definition information refers to information of a group composed of a plurality of item names and synonyms belonging to the group, and other synonyms belonging to the same group, and items included in the corrected text of the text correction information A speech output order control program that generates a corrected sentence including the matched item name, if there is a name that matches the item name of the sentence correction information. The attribute includes information indicating whether or not the same item is included in the detailed part of the output data for the output item,
In the determining process, based on the attribute of the output item, the output order of values output in a plurality of output items included in the specification part is set after the output order of values output in other output items. characterized in that, according to claim 1 Symbol placement of the audio output order control program. The correspondence relation accept the update instruction information, the update instruction on the basis, characterized in that it further comprises a process of updating the correspondence information, according to claim 1 or 2, wherein the audio output order control program. The control process includes a process of generating audio output data corresponding to the output data based on an output order of the plurality of values.
Processing for generating the audio output data in accordance with the data content item portion of the output data, characterized by omitting some of the voice output content item portion, one of the claims 1-3 1 The audio output order control program described in the section. The voice output order control program according to any one of claims 1 to 4 , wherein the data is data to be output as a form, and the definition information of the output data is form definition information. The attribute corresponding to each of a plurality of values included in the data is specified from the definition information of the output data including the plurality of values as output items,
The attribute is associated with the output order according to the semantic classification in the sentence structure determined according to the attribute, and the semantic classification is associated with the condition to be satisfied by the output item corresponding to the semantic classification. With reference to the storage unit in which the corresponding relationship information is stored, the output order of the plurality of values is determined according to the attribute corresponding to each of the plurality of values,
By referring to the sentence correction information indicating the sentence including the item and the character string to be complemented according to the item, and generating the corrected sentence including the output item based on the sentence correction information, the corrected sentence is included. Voice definition information
Applying the information of each item of the output data to the voice definition information to generate voice output data, the computer executes a process for controlling the output order when reading out the voice output data and outputting the voice,
The determining process refers to the correspondence information, and when the specific character string is arranged immediately before or immediately after the output item as the condition, the value that the semantic classification is output in the output item. Is determined to belong to the semantic classification, the output order is determined according to the semantic classification,
The process of generating the voice definition information refers to information of a group composed of a plurality of item names and synonyms belonging to the group, and other synonyms belonging to the same group, and items included in the corrected text of the text correction information A speech output order control method for generating a corrected sentence including a matched item name, if there is a name that matches the item name of the sentence correction information. An identification unit that identifies an attribute corresponding to each of a plurality of values included in the data from definition information of output data including the plurality of values as output items;
The attribute is associated with the output order according to the semantic classification in the sentence structure determined according to the attribute, and the semantic classification is associated with the condition to be satisfied by the output item corresponding to the semantic classification. A determination unit that determines the output order of the plurality of values according to attributes corresponding to the plurality of values, with reference to the storage unit in which the correspondence information is stored ;
By referring to the sentence correction information indicating the sentence including the item and the character string to be complemented according to the item, and generating the corrected sentence including the output item based on the sentence correction information, the corrected sentence is included. A generator that automatically generates voice definition information,
A control unit that applies the information of each item of the output data to the voice definition information to generate voice output data, and controls the output order when the voice output data is read out and voiced;
The determination unit refers to the correspondence relationship information, and when a specific character string is arranged immediately before or immediately after the output item as the condition, a value that the semantic classification is output in the output item is It is determined that it belongs to a semantic classification, the output order is determined according to the semantic classification,
The generation unit refers to information of a group composed of a plurality of item names and synonyms belonging to the group, and other synonyms belonging to the same group that match item names included in the corrected text of the text correction information If there is, the speech output order control device that generates a corrected sentence including the matched item name, assuming that it matches the item name of the sentence correction information.