Note: This lab assumes you have prior knowledge of the Speech to Text Service and/or have completed the Speech to Text lab. You will need a Bluemix account with a Node-RED application connected to the Speech to Text service to complete this lab.
##Introduction The speech to text customization feature has been introduced so the service can learn specific input that is unique to your use case. It is then trained based on pronunciations of unusual words within the input which it otherwise would not know, for example medical terms.
The language model customization interface of the Speech to Text service is currently only available for US English and Japanese.
This lab demonstrates how to create a sample customization.
The steps follow the usage model for working with Speech to text customization:
- Create a custom language model
- Add data from corpora to the custom language model
- Add words to the custom language model
- Train the custom language model
- Use the custom language model in a recognition request
For more information on the steps, view the documentation here.
Make sure you have installed the node-red-node-watson palette by clicking ‘Manage Palette’ under settings in the Node-RED editor. Click the ‘Install’ tab and search for node-red-node-watson then click install. Verify that the version is 0.4.37 or higher.
Ensure that in the package.json file for your Node-RED application the version for node-red-node-watson is set to at least 0.4.37. (Access this from the application overview page on Bluemix. Under 'Continous Delivery' in the bottom right hand corner, click 'Add Git Repo' if you haven't previously used the function or ‘Edit Code’. Navigate to the package.json file and update the line according to the image below. You will need to stop your application and restart it after saving the package.json file).
Start by adding 3 inject nodes, 3 speech to text custom builder nodes and one debug node. Join them together like the following example:
You do not need to configure the inject nodes.
The Create Customization node is configured like this:
You can set your own names for the Custom Model Name and Custom Model Description. This creates the customization using the desired model.
The List Customization node is configured like this:
This node allows you to view the customizations you have created.
The next step is to configure the debug node. The final customization node will be configured afterwards. Change the output to ‘complete msg object’ and press ‘Done’.
This allows you to view the full results of the inject nodes.
Deploy your application and click the inject node for ‘Create Customization’ and then ‘Get Customization’.
In the debug tab you should see two ‘msg : Object’s. Expand the bottom one and look for ‘customization_id’. This is your customization so make a note of the id as you will need it for the following steps.
Edit the final speech to text custom node by changing the ‘Detect’ field to ‘Get Customization’ and paste in the Customization ID you made a note of from the debug tab in the last step.
Deploy the application.
This completes the first step.
In this step, add 1 file inject node, 2 inject nodes and 3 more speech to text custom build nodes and join them to the debug node to look like the example below.
You do not need to do anything with the file inject or inject node.
In the speech to text node next to 'File Inject', name it ‘Add Corpus’, change the ‘Detect’ field to ‘Add Corpus’ and paste in the customization ID you used in the previous step. Give the corpus a name in the final field. Note: the Corpus Name will not work with spaces. You can choose whether to select the 'Allow Overwite' checkbox. If you leave it unchecked, you will not be able to add another Corpus to this customization.
In the second speech to text node editor, call this ‘Get Corpora’ and change the ‘Detect’ field to ‘Get Corpora’. Again paste in the Customization ID and select ‘Done’.
In the third speech to text node, call this ‘Train’ and change the ‘Detect’ field to ‘Detect’ and paste in the Customization ID.
Deploy your application.
Download the healthcare.txt. file from here to use as the Corpus.
Inject the healthcare.txt file into the file inject node before ‘Add Corpus’. You may have to wait a while for the corpus to be processed.
Click the timestamp before ‘Get Corpora’ and then press the timestamp before ‘Train’.
In the debug tab, open the object for ‘Get Corpora’. Open ‘Corpora’ and then ‘0: object’ until you find the status. The status needs to read ‘analyzed' before you can test out the function.
This completes the second step.
For this step, add 1 file inject node and 1 speech to text custom build node. Join these together and to the debug node.
You do not need to configure the file inject node. For the speech to text node, call it ‘Add Words’, change the ‘Detect’ field to ‘Add Words’ and paste in the Customization ID. Click ‘Done’ and deploy your app.
Download the words file (either .txt for Windows or .json for Mac OS). The file includes pronunciation for ‘NSAIDs’. You will use this file in step 5.
This completes step 3.
Click the timestamp for ‘Train’ and wait for it to finish processing.
You will now compare the difference between customized and non-customized results.
To do this, add 2 microphone nodes, 2 speech to text nodes and 2 debug nodes and join them together to look like the example below. This can be above or below the existing nodes. At this point it is a good idea to refresh the page as the speech to text node may not present the ‘Language Customization’ field which you will need. This is because the node fetches the models on the initial load to determine what is available. This does not get automatically updated.
You do not need to do anything with the microphone nodes. For the first speech to text node, name it 'Non-Customized', choose your language (either US English or Japanese) and select 'None' as the language customization. Select ‘Broadband Model’ for quality and check the continuous box. This will give you results without the customization.
For the second speech to text node, you must select US English or Japanese as these are currently only supported for the customization. Choose the language customization you created in step 1 from the drop down box. Select ‘Broadband Model’ for quality and check the continuous box.
For one of the debug tabs, set the output to msg.transcription. This will output the text converted from speech.
And the other debug tab output to msg.fullresult. This will output the full result of the speech to text function including confidence level and whether the output is final (showing true if final, false if not).
Deploy your application.
Test the speech to text non-customized function by speaking a line from the healthcare.txt file (preferably using words that are difficult to pronounce), as an example “Nonsteroidal anti-inflammatory drugs NSAIDs, such as aspirin and ibuprofen, are another common cause.” Take note of the output from Watson in the debug tab.
Repeat the question using the customized function.
Next, inject the words file you downloaded (either .txt or .json) in Step 3 to the 'Add Words' inject node and repeat the same sentence. The word 'NSAIDs' is elaborated on in the file.
If you compare the confidence rating in these results, the highest confidence should come once the words were added.
This completes the lab.
The complete flow is available here