JP6563882B2 - LABELING DEVICE, LABELING SYSTEM, LABELING METHOD, AND LABELING PROGRAM - Google Patents

Description

  The present invention relates to a label assigning device, a label assigning method, and a label assigning program for assigning a label for machine learning to target information.

  In recent years, it has been possible to create a large amount of data indicating the value of labels to be assigned to each of a large amount of target information using crowdsourcing that requests work such as unspecified number of people through the Internet. Has been done. For example, in the creation of learning data for machine learning, a label indicating whether or not a specific condition is satisfied is generally given to learning data as target information, but a label having multiple levels of strength Is also given. In Patent Document 1, a large number of images having intermediate characteristics between a person and a person are generated, and a lot of images are generated by allowing an operator to select a boundary between a person and a person through a radio button interface. A method for obtaining learning data is described.

Japanese Patent Laying-Open No. 2015-191426

  As described in Patent Document 1, in the method of assigning labels having multiple levels of strength to learning data using a radio button interface, the label granularity can be increased as the number of options increases. . However, since it is necessary for the worker to select one option from a plurality of options, the more options, the longer the time required for the worker's selection, which becomes inefficient. On the other hand, if the number of options is reduced, the required time is reduced, but there is a problem that the granularity of the label becomes rough.

  Accordingly, the present invention has been made in view of these points, and provides a labeling apparatus, a labeling method, and a labeling program capable of efficiently creating information having labels corresponding to multi-stage values. The purpose is to do.

  The label assigning apparatus according to the first aspect of the present invention includes a display control unit that displays target information to which a label is provided on a display unit provided with a touch panel, and a touch on a position corresponding to the target information on the touch panel. A receiving unit that receives an operation to perform, a specifying unit that specifies a value of a label corresponding to the target information, based on a touched position and a reference position that is predetermined for the target information; An assigning unit for assigning a label value to the target information.

  The display control unit displays a plurality of pieces of target information on the display unit, the receiving unit receives an operation of touching a position corresponding to each of the target information, and the specifying unit A label value corresponding to each of the plurality of pieces of the target information is specified based on the reference position predetermined for each of the plurality of pieces of the target information. Each of these may be assigned a value of the specified label.

  The display control unit displays set information including a plurality of pieces of the target information on the display unit, and the specifying unit extracts a plurality of pieces of the target information from the set information, the touched position, and the extracted plurality of pieces. Based on the predetermined reference position for each of the target information, a label value corresponding to each of the plurality of target information may be specified.

  The accepting unit accepts an operation of moving an area including a position corresponding to the target information while touching the touch panel, and the specifying unit adds the target information to the target information based on the trajectory of the operation and the reference position. A corresponding label value may be specified.

The accepting unit accepts a tap operation, a flick operation or a swipe operation as an operation to touch a position corresponding to the target information, and the specifying unit has a position corresponding to the tap operation, the flick operation or the swipe operation. The label value corresponding to the target information may be specified based on the reference position.
The display control unit may display information corresponding to a label value specified by the specifying unit at the touched position.

The label assigning device identifies an operation tendency of a user who operates the touch panel based on a plurality of label values specified in the past by the specifying unit, and sets a label value newly specified by the specifying unit. A correction unit that corrects based on the operation tendency may be further provided.
The display control unit may cause the display unit to display a score calculated based on the specified label value and the label value specified in advance by another user.
The display control unit displays target information to which a label relating to machine learning is given on the display unit, and the giving unit associates the target information with a label value specified for the target information. Teacher data may be generated.

  The label applying apparatus according to the second aspect of the present invention displays the target information to which the label is applied on the display unit, and causes the display unit to display a pointer indicating the input position by the input unit, Based on a receiving unit that accepts an operation of moving the pointer in an area including a position corresponding to the target information via an input unit, a trajectory of the operation, and a reference position predetermined for the target information And a specifying unit that specifies the value of the label corresponding to the target information, and an adding unit that adds the specified label value to the target information.

  A labeling system according to a third aspect of the present invention is a labeling system including a labeling device and a management device, and the management device transmits target information to which a label is to be given to the labeling device. A display control unit that displays the target information on a display unit provided with a touch panel, and a reception unit that receives an operation of touching a position corresponding to the target information on the touch panel. The management device or the label assigning device specifies a label value corresponding to the target information based on the touched position and a reference position predetermined for the target information. And a specifying unit for adding the specified label value to the target information.

A plurality of label assigning devices are provided, and the management device associates and stores the target information and a label value assigned to the target information, and a plurality of labels assigned to the same target information. And a calculation unit that calculates the reliability of the target information based on the value of.
The transmission unit may transmit target information whose calculated reliability is equal to or less than a predetermined threshold to the labeling apparatus to which the target information is not transmitted.

  The label assigning method according to the fourth aspect of the present invention corresponds to the target information in the step of displaying the target information to which the label is attached on a display unit provided with a touch panel, which is executed by a computer. A step of accepting an operation of touching a position, a step of specifying a value of a label corresponding to the target information based on the touched position and a reference position predetermined for the target information; Assigning the value of the label to the target information.

  The label assignment program according to the fifth aspect of the present invention is a display control unit that causes a computer to display target information to which a label is attached on a display unit provided with a touch panel, and a position corresponding to the target information in the touch panel. A receiving unit that receives an operation for touching, a specifying unit that specifies a value of a label corresponding to the target information, based on a touched position and a reference position predetermined for the target information, and The label value is added to the target information.

  According to the present invention, there is an effect that information having labels corresponding to multi-stage values can be efficiently created.

It is a figure which shows the outline | summary of the label provision system which concerns on 1st Embodiment. It is a figure which shows the structure of the management apparatus which concerns on 1st Embodiment. It is a figure which shows the structure of the label provision apparatus which concerns on 1st Embodiment. It is a sequence diagram which shows the flow of a process in the label provision system which concerns on 1st Embodiment. It is a figure which shows the example by which the target information which concerns on 1st Embodiment was displayed on the display part. It is a figure which shows the example by which the trace of the tracing operation which concerns on 1st Embodiment is displayed on the display part in real time. It is a figure which shows another example by which the target information which concerns on 1st Embodiment was displayed on the display part. It is a figure for demonstrating the distance of the reference | standard position which concerns on 1st Embodiment, and the touched position. It is a figure which shows the example which changed the display mode of the trace of the trace operation which concerns on 1st Embodiment based on the value of a label. It is a figure which shows the example which receives the position corresponding to object information by tap operation in 2nd Embodiment. It is a figure which shows the example which receives the position corresponding to object information by flick operation or swipe operation in 2nd Embodiment. It is a figure which shows the structure of the management apparatus which concerns on 3rd Embodiment. It is a figure which shows the example by which the score which concerns on 4th Embodiment was displayed on the display part.

<First Embodiment>
[Outline of labeling system S]
FIG. 1 is a diagram showing an overview of a labeling system S according to the first embodiment. As shown in FIG. 1, the labeling system S includes a management device 1 and a labeling device 2. The label assignment system S is a system that assigns labels corresponding to multi-stage values to target information. Here, the target information according to the first embodiment is information to which a label relating to machine learning is given, for example. The target information and the value of the label are used as teacher data for machine learning. However, the present invention is not limited to this, and is used for other purposes such as using the assigned label as statistical information as it is for questionnaire results. May be.

  The management apparatus 1 is a server connected to the label assignment apparatus 2 via a communication network. The label assigning device 2 is a mobile terminal such as a smartphone. The label assigning device 2 is used by a user who performs label assignment work on target information as work related to crowdsourcing. In FIG. 1, only one label attaching device 2 is displayed, but a plurality of label attaching devices 2 are assumed to exist.

  In the first embodiment, the management device 1 transmits the target information to the label applying device 2 in response to a request for acquiring the target information from the label applying device 2 ((1) in FIG. 1). The label assigning device 2 displays the received target information on a display unit provided with a touch panel ((2) in FIG. 1), and accepts an operation of touching a position corresponding to the target information ((3) in FIG. 1). . Based on the touched position, the label assigning device 2 identifies label values corresponding to multi-stage values ((4) in FIG. 1), and assigns the identified label values to the target information. By doing in this way, the label assignment apparatus 2 can efficiently create information having labels corresponding to multi-stage values based on a simple operation of touching a position corresponding to target information. .

The label assigning device 2 transmits the specified label value and the target information to the management device 1 ((5) in FIG. 1). Upon receiving the label value and the target information, the management device 1 stores the label value in association with the target information ((6) in FIG. 1). Thereby, the management apparatus 1 can give the label which shows the intensity | strength of several steps to object information.
Hereinafter, configurations of the management device 1 and the label applying device 2 will be described.

[Configuration Example of Management Device 1]
First, the configuration of the management apparatus 1 will be described. FIG. 2 is a diagram illustrating a configuration of the management apparatus 1 according to the first embodiment. The management device 1 includes a storage unit 11 and a control unit 12.

  The storage unit 11 is, for example, a ROM and a RAM. The storage unit 11 stores various programs for causing the management device 1 to function. For example, the storage unit 11 stores a management program that causes the management apparatus 1 to function as the transmission unit 121, the reception unit 122, and the storage control unit 123. Further, the storage unit 11 stores target information and the value of a label given to the target information.

  The control unit 12 is a CPU, for example. The control unit 12 controls functions related to the management device 1 by executing various programs stored in the storage unit 11. The control unit 12 includes a transmission unit 121, a reception unit 122, and a storage control unit 123. Details of these functions will be described later.

[Configuration Example of Labeling Device 2]
Then, the structure of the label provision apparatus 2 is demonstrated. FIG. 3 is a diagram illustrating a configuration of the label applying apparatus 2 according to the first embodiment. The label assigning device 2 includes a display unit 21, an input unit 22, a storage unit 23, and a control unit 24.

The display unit 21 includes, for example, a liquid crystal display or an organic EL (Electro-Luminescence) display. The display unit 21 displays target information, information indicating a label value corresponding to the target information, and the like according to the control of the control unit 24.
The input unit 22 is, for example, a touch panel or a button configured by a contact sensor or the like arranged corresponding to the display unit 21 and receives an operation input from the user of the label applying device 2.

  The storage unit 23 is, for example, a ROM and a RAM. The storage unit 23 stores various programs for causing the label attaching device 2 to function. For example, the storage unit 23 functions as the control unit 24 of the label applicator 2 as an acquisition unit 241, a display control unit 242, a reception unit 243, a specification unit 244, a correction unit 245, and a transmission unit 246 as an addition unit, which will be described later. The labeling program to be executed is stored.

  The control unit 24 is a CPU, for example. The control unit 24 controls functions related to the labeling device 2 by executing various programs stored in the storage unit 23. The control unit 24 includes an acquisition unit 241, a display control unit 242, a reception unit 243, a specification unit 244, a correction unit 245, and a transmission unit 246.

[Flow of processing in the labeling system S]
Subsequently, each function provided in the control unit 12 provided in the management device 1 and the control unit 24 provided in the label applying device 2 will be described with reference to a sequence diagram showing a processing flow in the label applying system S. FIG. 4 is a sequence diagram showing the flow of processing in the labeling system S according to the first embodiment.

First, the acquisition unit 241 of the label assigning apparatus 2 transmits an acquisition request for target information to the management apparatus 1 (S10).
The transmission unit 121 of the management device 1 does not transmit the target information stored in the storage unit 11 to the labeling device 2 in response to receiving the target information acquisition request from the labeling device 2. Target information is acquired (S20).

  Here, the target information is, for example, a word or an image. The transmission unit 121 acquires set information including a plurality of pieces of target information. For example, the transmission unit 121 acquires a sentence including a plurality of words as set information. The transmission unit 121 transmits a plurality of pieces of target information to the label attaching device 2 by transmitting the acquired set information (S30).

The acquisition unit 241 of the label assigning device 2 acquires aggregate information including a plurality of pieces of target information transmitted from the management device 1.
Subsequently, the display control unit 242 of the label assigning apparatus 2 displays target information to which a label relating to machine learning is provided on the display unit 21 provided with the touch panel that constitutes the input unit 22 (S40).

  FIG. 5 is a diagram illustrating an example in which the target information according to the first embodiment is displayed on the display unit 21. For example, as illustrated in FIG. 5, the display control unit 242 causes the display unit 21 to display sentences Sa and Sb as collective information including a plurality of pieces of target information.

  In addition, the display control unit 242 displays information indicating the scale of the label to be assigned to each of the words constituting the sentences Sa and Sb. In the example illustrated in FIG. 5, as the scale information M indicating the scale of the label, the character strings “Yes”, “No”, and arrows indicating whether the word is regarded as positive are displayed.

  Returning to FIG. 4, the accepting unit 243 accepts an operation of touching a position corresponding to the target information on the touch panel constituting the input unit 22 from the user of the label applying device 2 (S <b> 50). Specifically, the accepting unit 243 accepts an operation of touching a position corresponding to each of a plurality of pieces of target information corresponding to the collective information on the touch panel. More specifically, the accepting unit 243 accepts an operation for moving an area including a position corresponding to the target information while touching the touch panel, that is, an operation for tracing the positions corresponding to a plurality of target information from the user. In the example illustrated in FIG. 5, the receiving unit 243 receives an operation of tracing the positions corresponding to a plurality of pieces of target information in the areas Aa and Ab corresponding to the sentences Sa and Sb as the set information. For example, an operation of tracing the position above the sentence Sa in the area Aa corresponds to positively capturing each word constituting the sentence Sa. Further, the operation of tracing the position below the sentence Sa in the area Aa corresponds to negatively capturing each word constituting the sentence Sa.

  Here, the display control unit 242 may cause the display unit 21 to display a locus of an operation of tracing the touch panel (hereinafter also referred to as a tracing operation) in real time. FIG. 6 is a diagram illustrating an example in which the trace T of the tracing operation according to the first embodiment is displayed on the display unit 21 in real time. By doing in this way, the label provision apparatus 2 can make a user grasp | ascertain the position where the touch operation was detected in the display part 21. FIG.

  In the above description, an example in which the set information is a sentence has been described. However, the set information is not limited to a sentence, and may be a plurality of images. For example, the acquisition unit 241 may acquire a plurality of temporally continuous images extracted from a moving image or the like from the management device 1 as collective information. The acquisition unit 241 may acquire a moving image from the management device 1 and extract a plurality of temporally continuous images from the moving image as collective information.

  FIG. 7 is a diagram illustrating another example in which the target information according to the first embodiment is displayed on the display unit 21. As illustrated in FIG. 7A, the display control unit 242 displays a plurality of temporally continuous images on the display unit 21 as collective information including a plurality of pieces of target information. In the example shown in FIG. 7, it can be confirmed that images of the same person's face and different expressions are displayed. In the example shown in FIG. 7, it can be confirmed that the character strings “pleasant”, “unpleasant”, and arrows indicating whether the expression is comfortable are displayed as the scale of the label.

  In this case, the accepting unit 243 accepts an operation for tracing a position corresponding to a plurality of images. Then, the display control unit 242 displays the trace of the tracing operation on the display unit 21 in real time as shown in FIG. 7B.

  Returning to FIG. 4, the specifying unit 244 specifies the value of the label corresponding to the target information based on the touched position and a reference position predetermined for the target information (S <b> 60). Specifically, the specifying unit 244 specifies a label value corresponding to each of the plurality of pieces of target information based on the touched position and a reference position predetermined for each of the plurality of pieces of target information. To do.

  More specifically, the specifying unit 244 extracts a plurality of pieces of target information from the set information, and a predetermined reference for each of the touched position indicated by the trace operation trace and each of the plurality of pieces of extracted target information. Based on the position, the value of the label corresponding to each of the plurality of pieces of target information is specified. As illustrated in FIG. 5, when the set information is a sentence, the specifying unit 244 performs morphological analysis, so that the importance of a sentence such as a noun or an adjective is determined as a plurality of pieces of target information from the set information. Extract relatively high words. For example, the identification unit 244 extracts “a company”, “mobile phone”, and “cool” from the sentence Sa shown in FIG. The specifying unit 244 may extract all the words included in the sentence by performing morphological analysis.

  Subsequently, the specifying unit 244 calculates a distance between each reference position of the extracted plurality of words in the sentence displayed on the display unit 21 and a position touched corresponding to the word. FIG. 8 is a diagram for explaining the distance between the reference position and the touched position. As shown in FIG. 8, the reference position is a position corresponding to a line segment L connecting the centers of a plurality of characters constituting the sentence.

  The specifying unit 244 determines a position having the longest distance from the position of the line segment corresponding to the extracted word among the positions on the trajectory of the tracing operation as a reference position and a position touched corresponding to the word. Is calculated as a distance. In the example illustrated in FIG. 8, the specifying unit 244 specifies a trajectory included in the region Ac corresponding to the word “cool” in the trajectory T. Then, the specifying unit 244 calculates a distance D between the position on the specified locus and the farthest from the line segment L and the reference position of the word “cool”. The specifying unit 244 calculates the distance corresponding to each extracted word in the same procedure.

  The specifying unit 244 specifies the label value in multiple stages based on the calculated distance. For example, when the distance of the word “cool” is 6.3425, the specifying unit 244 sets the value of the label of the word “cool” to 6.3425.

  The display control unit 242 may display information corresponding to the label value specified by the specifying unit 244 at the touched position. For example, when displaying the trace of the tracing operation on the display unit 21 in real time, the display control unit 242 may change the display mode of the trace based on the value of the label.

  FIG. 9 is a diagram illustrating an example in which the display mode of the trace T of the tracing operation according to the first embodiment is changed based on the value of the label. In the example shown in FIG. 9, when the label value is a relatively high value, the locus T is displayed thick, and when the label value is a relatively low value, the locus T is displayed as a broken line. Has been. By doing in this way, the label provision apparatus 2 can make a user grasp | ascertain the specific condition of the value of a label.

  In the example illustrated in FIG. 9, the display control unit 242 displays the trajectory T thickly or with a broken line according to the value of the label, but this is not restrictive. The display control unit 242 may change the color of the locus T according to the value of the label.

  Returning to FIG. 4, the correcting unit 245 specifies the operation tendency of the user who operates the touch panel constituting the input unit 22 based on the values of the plurality of labels specified in the past by the specifying unit 244, and the specifying unit 244 The newly specified label value is corrected based on the operation tendency (S70).

  For example, the correcting unit 245 specifies the distribution of the label values based on a plurality of label values specified in the past by the specifying unit 244. Then, the correction unit 245 corrects the label value specified based on the dispersion value. For example, the correction unit 245 obtains the average value of the variances calculated in the plurality of label applying devices 2 from the management device 1, and the label value is decreased when the variance value is relatively high. If the variance value is relatively low, the label value is corrected to be large. By doing in this way, the label between the different users who operate each of the some label provision apparatus 2 can be handled uniformly. Here, the display control unit 242 may change the display mode of the tracing operation trace based on the corrected label value.

  The correcting unit 245 may correct the label value by changing the position of the trajectory T on the display unit 21 based on the trajectory T and the set information. For example, among users, there is a user whose trajectory of the tracing operation is shifted left and right. Therefore, the correction unit 245 may specify the amount of deviation of the tracing operation trajectory to the left and right based on the distance between the touch position calculated based on the tracing operation and the reference position and the position of the target information. Then, the correction unit 245 may correct the label value by changing the position of the trajectory T on the display unit 21 based on the shift amount.

  In the present embodiment, the correction unit 245 corrects the label value newly specified by the specifying unit 244 based on the operation tendency. However, the present invention is not limited to this. The label applying device 2 may not include the correcting unit 245.

  The transmission unit 246 manages the label values of the plurality of pieces of target information specified by the specifying unit 244 and corrected by the correction unit 245, the plurality of pieces of target information, and the identification information of the label applying device 2. It transmits to the apparatus 1 (S80). For example, the transmission unit 246 generates teacher data in which the target information is associated with the label value specified and corrected for the target information, and the teacher data and the identification information of the label providing device 2 Is transmitted to the management apparatus 1. Thereby, the transmission unit 246 causes the management device 1 to assign the specified label value to the target information.

The receiving unit 122 receives target information, a label value, and identification information of the label applying device 2 from the label applying device 2.
The storage control unit 123 stores the label value in the storage unit 11 (S90). Specifically, the storage control unit 123 receives the target information, the label value, and the identification information of the label applying device 2. Then, the storage control unit 123 stores the target information, the label value, and the identification information of the label applying device 2 in the storage unit 11 in association with each other.

[Effect in the first embodiment]
As described above, in the labeling system S according to the first embodiment, the labeling device 2 receives an operation of touching a position corresponding to the target information displayed on the display unit 21 on the touch panel, and the touched position; Based on a predetermined reference position for the target information, a label value corresponding to the target information is specified, and the specified label value is assigned to the target information. By doing in this way, the label provision apparatus 2 can produce the label corresponding to a multistep value efficiently based on simple operation of touching the position corresponding to object information in a touch panel.

Second Embodiment
Next, the second embodiment will be described. The label applying apparatus 2 according to the second embodiment is different from the first embodiment in that the position corresponding to the target information is received by an operation different from the operation of tracing the position corresponding to the plurality of target information. Hereinafter, description of the same parts as those in the first embodiment will be omitted as appropriate.

  In 2nd Embodiment, the reception part 243 receives tap operation as touch operation of the position corresponding to object information. FIG. 10 is a diagram illustrating an example in which a position corresponding to target information is received by a tap operation in the second embodiment. For example, as shown in FIG. 10A, the display control unit 242 displays lines L1 and L2 corresponding to label values for a sentence as set information. Here, it is assumed that the values of the labels corresponding to the lines L1 and L2 are values indicating neutrality (for example, 0). And the reception part 243 receives the tap operation in the area | region above and below the lines L1 and L2 as operation which touches the position corresponding to object information. For example, when the user positively captures “cool”, the user performs an operation of tapping the upper area “Ad” of “cool” as illustrated in FIG. When the reception unit 243 receives an operation of tapping the area Ad, the display control unit 242 changes the shape of the line L1 so that the line L1 passes through the touched position as illustrated in FIG. 10B. .

  In the second embodiment, the reception unit 243 receives a flick operation or a swipe operation as a touch operation at a position corresponding to the target information. FIG. 11 is a diagram illustrating an example in which a position corresponding to target information is received by a flick operation or a swipe operation in the second embodiment. As shown in FIG. 11A, the display control unit 242 displays lines L1 and L2 corresponding to the label values for the sentence as the set information. And the reception part 243 receives operation which flicks or swipes to the area | region above and below lines L1, L2 from the position where lines L1, L2 are displayed as operation which touches the position corresponding to object information. For example, when the user positively captures “cool”, as shown in FIG. 11A, the user flicks in the direction of the broken line arrow, starting from the position corresponding to “cool” in the line L1. Or perform a swipe operation. When the accepting unit 243 accepts an operation of flicking or swiping, the display control unit 242 changes the shape of the line L1 so that the line L1 passes through the touched position, as illustrated in FIG.

  The specifying unit 244 specifies the value of the label corresponding to the target information based on the position corresponding to the tap operation, the flick operation, or the swipe operation and the reference position. For example, when the tap operation is performed, the specifying unit 244 determines the value of the label based on the position where the tap operation is performed and the distance between the position and the line displayed corresponding to the target information. Is identified. In addition, when a flick operation or a swipe operation is performed, the specifying unit 244 specifies a label value based on the position where the operation ends and the distance between the line displayed corresponding to the target information. .

[Effects of Second Embodiment]
As described above, in the labeling system S according to the second embodiment, the labeling device 2 accepts a tap operation, a flick operation, or a swipe operation as an operation for touching a position corresponding to the target information, and the tap operation, flick Based on the position corresponding to the operation or swipe operation and the reference position, the value of the label corresponding to the target information is specified. By doing in this way, the label provision apparatus 2 can specify the value of a label also by tap operation, flick operation, or swipe operation different from the touch operation which traces object information.

<Third Embodiment>
[Calculate the reliability of the label value of the target information]
Subsequently, the third embodiment will be described. The label assignment system S according to the third embodiment is different from the first embodiment in that the management device 1 calculates the reliability of the label value of the target information.
FIG. 12 is a diagram illustrating a configuration of the management apparatus 1 according to the third embodiment. As illustrated in FIG. 12, the control unit 12 of the management device 1 further includes a calculation unit 124.

  The calculation unit 124 calculates the reliability based on the label values given to the same target information received from the plurality of label assignment devices 2. For example, the calculation unit 124 calculates the variance as the reliability based on the value of the label given to the same target information. Then, the calculation unit 124 calculates the reliability based on the calculated variance. For example, the calculation unit 124 calculates the reliability so that the reliability is low when the variance value is high and the reliability is high when the variance value is low.

  When the transmission unit 121 receives an acquisition request for target information from the labeling device 2 and acquires target information to be transmitted to the labeling device 2 from the storage unit 11, the calculated reliability is a target with a predetermined threshold value or less. Information, that is, target information to which a label with low reliability is attached is acquired from the storage unit 11. And the transmission part 121 transmits the said object information to the label provision apparatus 2 which transmitted the acquisition request. Thereby, the transmission part 121 transmits the target information to which the label with low reliability is attached | subjected to the label provision apparatus 2 to which the said target information is not transmitted.

[Effect in the third embodiment]
As described above, in the label assignment system S according to the third embodiment, the management device 1 calculates the reliability of the target information based on the values of a plurality of labels assigned to the same target information. By doing in this way, the management apparatus 1 can identify the target information in which the polarity of the label is divided and the target information having a variation in the value attached to the label. And the management apparatus 1 transmits the said target information to the label provision apparatus 2, and can re-determine reliability based on the value of the label newly provided with respect to the said target information. In addition, the management apparatus 1 can determine the reliability more accurately by transmitting the target information to the plurality of label applying apparatuses 2 and acquiring the label value.

<Fourth embodiment>
[Display score based on specified label value]
Subsequently, a fourth embodiment will be described. The label assigning apparatus 2 according to the fourth embodiment is different from the first embodiment in that the score is calculated based on the specified label value and the score is displayed on the display unit 21.

In the fourth embodiment, the acquisition unit 241 acquires target information from the management apparatus 1 and an average value of label values specified in advance by a plurality of other users for the target information.
The display control unit 242 calculates the score based on the value of the label specified by the specifying unit 244 and the value of the label specified by another user. Specifically, the display control unit 242 calculates the score so that the smaller the difference between the specified label value and the average value of the label values specified by other users is, the higher the score is. . Then, the display control unit 242 causes the display unit 21 to display the calculated score. FIG. 13 is a diagram illustrating an example in which the score according to the fourth embodiment is displayed on the display unit 21. In FIG. 13, it can be confirmed that the object O indicating the calculated score is displayed. The display control unit 242 may calculate points for each of the plurality of pieces of target information, and display the total value of the points on the display unit 21.

  The transmission unit 246 transmits the calculated score and the identification information of the label applying device 2 to the management device 1. The management device 1 stores the received score and the identification information of the label applying device 2 in the storage unit 11 in association with each other.

[Effects of the fourth embodiment]
As described above, in the label assigning system S according to the fourth embodiment, the label assigning device 2 calculates the score calculated based on the specified label value and the label value specified in advance by another user. It is displayed on the display unit 21. By doing in this way, the user of the label provision apparatus 2 will perform label provision operation so that a score may become high. Thereby, the label assigning device 2 can assign an appropriate label value to the target information.

  In the present embodiment, the acquiring unit 241 may acquire the highest score from the scores received from the plurality of label applying devices 2 from the management device 1. The display control unit 242 may display the acquired score on the display unit 21 and may display the highest score on the display unit 21. By doing in this way, the user of the label provision apparatus 2 is conscious of the score of another user, and performs touch operation aiming at a score higher than the said score. Thereby, the label assigning device 2 can assign a more appropriate label value to the target information.

  As mentioned above, although this invention was demonstrated using embodiment, the technical scope of this invention is not limited to the range as described in the said embodiment. It will be apparent to those skilled in the art that various modifications or improvements can be added to the above embodiment.

  For example, in the first embodiment, the label applying apparatus 2 specifies the value of the label to be applied to the target information, but the present invention is not limited to this. For example, even if the label assigning device 2 transmits the target information and the position information indicating the touched position on the touch panel to the management device 1, the management device 1 specifies the value of the label based on the position information. Good.

  Moreover, in 1st Embodiment, although the label provision apparatus 2 shall be portable terminals, such as a smart phone, and target information was displayed on the display part 21 provided with the touch panel, it is not restricted to this. The label attaching device 2 may be a desktop personal computer or a notebook personal computer using a mouse as the input unit 22. In this case, the display control unit 242 may display target information on the display unit 21 and display a pointer indicating an input position by the input unit 22 on the display unit 21. Then, the receiving unit 243 receives an operation of moving the pointer in an area including the position corresponding to the target information via the input unit 22, and the specifying unit 244 is predetermined for the operation trajectory and the target information. The label value corresponding to the target information may be specified based on the reference position. In this way, a desktop personal computer or notebook personal computer can be operated as the labeling device 2.

  Further, in the first embodiment, as shown in FIG. 7, the label attaching device 2 displays a plurality of continuous images extracted from the moving image, and accepts an operation of tracing the position corresponding to the images. Not limited to. For example, the labeling apparatus 2 reproduces and displays a moving image on the display unit 21, and on the touch panel constituting the input unit 22, an operation for indicating a scale for the moving image (for example, up and down on the touch panel) (Tracing operation) may be accepted. Then, the label assigning device 2 may specify a label value based on the position traced in each of the reproduction times of the moving image, and assign the label value to the image corresponding to the reproduction time.

  In particular, the specific embodiments of the distribution / integration of the devices are not limited to those illustrated above, and all or a part thereof may be arbitrarily selected according to various additions or according to the functional load. It can be configured to be functionally or physically distributed and integrated in units.

DESCRIPTION OF SYMBOLS 1 ... Management apparatus, 11 ... Storage part, 12 ... Control part, 121 ... Transmission part, 122 ... Reception part, 123 ... Storage control part, 124 ... Calculation part, DESCRIPTION OF SYMBOLS 2 ... Label applicator, 21 ... Display part, 22 ... Input part, 23 ... Memory | storage part, 24 ... Control part, 241 ... Acquisition part, 242 ... Display control part 243 ... reception unit, 244 ... identification unit, 245 ... correction unit, 246 ... transmission unit, S ... labeling system

Claims (21)

A display control unit that displays aggregate information including a plurality of pieces of target information to which a label is attached on a display unit provided with a touch panel;
In the touch panel, a reception unit that receives an operation of touching a position corresponding to each of the plurality of pieces of target information;
A plurality of the target information is extracted from the collective information, and each of the plurality of target information is determined based on a touched position and a reference position predetermined for each of the extracted plurality of target information . A specific part that identifies the value of the corresponding label;
An assigning unit for assigning a value of the identified label to a plurality of the target information;
A labeling device comprising:   A display control unit for displaying target information to which a label is attached on a display unit provided with a touch panel;
  In the touch panel, a reception unit that receives an operation of touching a position corresponding to the target information;
  A specifying unit that specifies a value of a label corresponding to the target information based on the touched position and a reference position predetermined for the target information;
  An assigning unit for assigning the specified label value to the target information;
  With
  The display control unit displays information corresponding to the value of the label specified by the specifying unit at the touched position.
  Labeling device.   A display control unit for displaying target information to which a label is attached on a display unit provided with a touch panel;
  In the touch panel, a reception unit that receives an operation of touching a position corresponding to the target information;
  A specifying unit that specifies a value of a label corresponding to the target information based on the touched position and a reference position predetermined for the target information;
  An assigning unit for assigning the specified label value to the target information;
  With
  The display control unit causes the display unit to display a score calculated based on the specified label value and the label value specified in advance by another user.
  Labeling device. The accepting unit accepts an operation for moving an area including a position corresponding to the target information while touching the touch panel,
The specifying unit specifies a value of a label corresponding to the target information based on the trajectory of the operation and the reference position;
The label applicator according to any one of claims 1 to 3. The accepting unit accepts a tap operation, a flick operation or a swipe operation as an operation for touching a position corresponding to the target information,
The specifying unit specifies a value of a label corresponding to the target information based on the position corresponding to the tap operation, the flick operation or the swipe operation, and the reference position.
The label applicator according to any one of claims 1 to 3.   Based on a plurality of label values specified in the past by the specifying unit, an operation tendency of a user who operates the touch panel is specified, and a label value newly specified by the specifying unit is determined based on the operation tendency. A correction unit for correcting
  The label applicator according to any one of claims 1 to 5.   The display control unit displays target information to which a label relating to machine learning is given on the display unit,
  The assigning unit generates teacher data in which the target information is associated with a label value specified for the target information.
  The label applicator according to any one of claims 1 to 6.   A display control unit that displays aggregate information including a plurality of pieces of target information to which a label is attached on the display unit, and displays a pointer indicating an input position by the input unit on the display unit;
  A receiving unit that receives an operation of moving the pointer in a region including a position corresponding to each of the plurality of pieces of target information via the input unit;
  A plurality of the target information is extracted from the set information, and each of the plurality of target information is determined based on the trajectory of the operation and a reference position predetermined for each of the extracted plurality of the target information. A specific part that identifies the value of the corresponding label;
  An assigning unit for assigning a value of the identified label to a plurality of the target information;
  A labeling device comprising:   A display control unit that displays target information to which a label is attached on the display unit, and displays a pointer indicating an input position by the input unit on the display unit;
  A receiving unit that receives an operation of moving the pointer in a region including a position corresponding to the target information via the input unit;
  A specifying unit that specifies a value of a label corresponding to the target information based on the trajectory of the operation and a reference position predetermined for the target information;
  An assigning unit for assigning the specified label value to the target information;
  With
  The display control unit displays information corresponding to the value of the label specified by the specifying unit at a position where the pointer is operated.
  Labeling device. A display control unit that displays target information to which a label is attached on the display unit, and displays a pointer indicating an input position by the input unit on the display unit;
A receiving unit that receives an operation of moving the pointer in a region including a position corresponding to the target information via the input unit;
A specifying unit that specifies a value of a label corresponding to the target information based on the trajectory of the operation and a reference position predetermined for the target information;
An assigning unit for assigning the specified label value to the target information;
With
The display control unit causes the display unit to display a score calculated based on the specified label value and the label value specified in advance by another user.
Labeling device. A labeling system comprising a labeling device and a management device,
The management device
A transmission unit that transmits aggregate information including a plurality of pieces of target information to which labels are attached to the label attaching device;
The labeling device is:
A display control unit for displaying the target information on a display unit provided with a touch panel;
A receiving unit that receives an operation of touching a position corresponding to each of the plurality of pieces of target information in the touch panel;
The management device or the labeling device is
A plurality of the target information is extracted from the collective information, and each of the plurality of target information is determined based on a touched position and a reference position predetermined for each of the extracted plurality of target information . A specific part that identifies the value of the corresponding label;
And an adding unit that assigns the specified label value to the plurality of pieces of the target information.
Labeling system.   A labeling system comprising a labeling device and a management device,
  The management device
  A transmission unit that transmits target information to which a label is attached to the label attaching device;
  The labeling device is:
  A display control unit for displaying the target information on a display unit provided with a touch panel;
  A receiving unit that receives an operation of touching a position corresponding to the target information in the touch panel;
  The management device or the labeling device is
  A specifying unit that specifies a value of a label corresponding to the target information based on the touched position and a reference position predetermined for the target information;
  A granting unit for granting the specified label value to the target information;
  The display control unit displays information corresponding to the value of the label specified by the specifying unit at the touched position.
  Labeling system.   A labeling system comprising a labeling device and a management device,
  The management device
  A transmission unit that transmits target information to which a label is attached to the label attaching device;
  The labeling device is:
  A display control unit for displaying the target information on a display unit provided with a touch panel;
  A receiving unit that receives an operation of touching a position corresponding to the target information in the touch panel;
  The management device or the labeling device is
  A specifying unit that specifies a value of a label corresponding to the target information based on the touched position and a reference position predetermined for the target information;
  A granting unit for granting the specified label value to the target information;
  The display control unit causes the display unit to display a score calculated based on the specified label value and the label value specified in advance by another user.
  Labeling system.   A plurality of the labeling devices are provided,
  The management device
  A storage unit that associates and stores the target information and a value of a label given to the target information;
  A calculation unit that calculates the reliability of the target information based on the values of a plurality of labels attached to the same target information;
  The labeling system according to any one of claims 11 to 13.   The transmission unit transmits target information whose calculated reliability is equal to or lower than a predetermined threshold to the labeling device to which the target information is not transmitted.
  The labeling system according to claim 14.   The computer runs,
  Displaying aggregate information including a plurality of target information to which a label is attached on a display unit provided with a touch panel;
  Receiving an operation of touching a position corresponding to each of the plurality of pieces of target information on the touch panel;
  A plurality of the target information is extracted from the collective information, and each of the plurality of target information is determined based on a touched position and a reference position predetermined for each of the extracted plurality of target information. Identifying the corresponding label value;
  Assigning the identified label values to a plurality of the target information;
  A labeling method comprising:   The computer runs,
  Displaying target information to which a label is attached on a display unit provided with a touch panel;
  Receiving an operation of touching a position corresponding to the target information on the touch panel;
  Identifying a value of a label corresponding to the target information based on the touched position and a reference position predetermined for the target information;
  Assigning the value of the identified label to the target information;
  Displaying information corresponding to the value of the identified label at the touched position;
  A labeling method comprising:   The computer runs,
  Displaying target information to which a label is attached on a display unit provided with a touch panel;
  Receiving an operation of touching a position corresponding to the target information on the touch panel;
  Identifying a value of a label corresponding to the target information based on the touched position and a reference position predetermined for the target information;
  Assigning the value of the identified label to the target information;
  Displaying the score calculated based on the specified label value and the label value specified in advance by another user on the display unit;
  A labeling method comprising:   Computer
  A display control unit that displays aggregate information including a plurality of pieces of target information to which labels are attached on a display unit provided with a touch panel;
  In the touch panel, a reception unit that receives an operation of touching a position corresponding to each of the plurality of pieces of target information;
  A plurality of the target information is extracted from the collective information, and each of the plurality of target information is determined based on a touched position and a reference position predetermined for each of the extracted plurality of target information. A specific part for identifying the value of the corresponding label, and
  An assigning unit for assigning the specified label value to a plurality of pieces of the target information;
  Labeling program to function as.   Computer
  A display control unit for displaying target information to which a label is attached on a display unit provided with a touch panel;
  In the touch panel, a reception unit that receives an operation of touching a position corresponding to the target information;
  A specifying unit that specifies a value of a label corresponding to the target information based on the touched position and a reference position predetermined for the target information; and
  An assigning unit for assigning the identified label value to the target information;
  Function as
  The display control unit displays information corresponding to the value of the label specified by the specifying unit at the touched position.
  Labeling program.   Computer
  A display control unit for displaying target information to which a label is attached on a display unit provided with a touch panel;
  In the touch panel, a reception unit that receives an operation of touching a position corresponding to the target information;
  A specifying unit that specifies a value of a label corresponding to the target information based on the touched position and a reference position predetermined for the target information; and
  An assigning unit for assigning the identified label value to the target information;
  Function as
  The display control unit causes the display unit to display a score calculated based on the specified label value and the label value specified in advance by another user.
  Labeling program.

Images