NLP

NLP stands for Natural Language Processing and is the ability for a machine to be able to take an unstructured piece of text, such as a sentence of paragraph and process it in such a way as to understand the meaning behind it. This is a technique that falls under the umbrella term of Artificial Intelligence.

Within a digital assistant project we use NLP to understand what the user has typed. This is very important in terms of determining the users Intent, and any Entities contained within the message.

Using NLP can make a massive difference to a users experience when it comes to digital assistants. Rather than having to navigate through a series of menus, a user can express exactly what they need and be directed exactly to the content they were after.

In particular, when thinking about a digital assistant, we use NLP to perform a type of task called Classification, and in particular Intent Classification. It is a supervised learning task in that the system can not classify the intent correctly without some information provided to it in the first place. This supervised process has the following steps :

• Generate a data set containing a wide and diverse set of things that a typical user might say.

• Apply a label to each piece of data to help the machine understand what that sentence means. This label will be the intent name.

• With the set of data now labelled we pass the information through an NLP process which generates a machine learning model. This model is unique and based on the training data set that has been provided.

Once you have created a model we can then evaluate it by providing new phrases ( or Utterances ) to it for classification. This phrase will then be evaluated against each label that is within the model. The process will provide a score for each label, between 0 and 1, where 1 is essentially 100% confident in the classification. The accuracy of a model will be a measure of how many times a new utterance is correctly identified as the right intent. In this case the intent identified is the intent that has the highest confidence value.

To get started with NLP , you can watch this video to better understand some of the terminology used in the NLP world.