Including various numbers as part of sentence?

WARNING: You currently have Javascript disabled!
This website will not function correctly without Javascript enabled.

djskope

11 discussion posts

Hi, thanks for the excellent program!

I have started writing a profile which is working very well with GNU Backgammon's 'Command' input box. For example, I can roll the dice with my voice and choose to cube or drop with my voice.

This is huge improvement over the hundreds of mouse clicks normally required to complete a backgammon match!

One other thing is missing: the 'Command' box will allow you to specify your moves in text

i.e. the sentence

move 8 4 6 4

Which (if you have rolled 4-2) will move two checkers to the 4 point from the 8 and the 6

My problem is, how do I get voicebox to accept such input?

Is there a way to get it to recognise numbers as part of the command?

N.B. 1 - 24 (plus the words 'bar' and 'off') are valid 'point' numbers in GNU backgammon.

I have attached a screenshot if that helps.

I think this could be quite useful for blind people who can learn to play backgammon (I know at least one). I plan to share my profile here once it is finished.

Many thanks for any light you can shed!

• Attachment: GNU Backgammon (2016-10-10@08.02.14).png [879,394 bytes]

GNU Backgammon (2016-10-10@08.02.14).png

Oct 10, 2016 (modified Oct 10, 2016) • #1

We're planning to add this functionality in a future version, but in the meantime, it may be possible with a scripted function. We'll try to create a sample one for you next week.

Thanks!

Oct 11, 2016 • #2

djskope

11 discussion posts

Thanks - that's great news.

I should also give some thought to audible feedback that the move (with the variable numbers) was successfully executed.

I guess this would best be a confirmation sentence like
"Moved 8 4 6 4"

I'm hoping that this will somehow be possible with your 'text to speech' action.

Argh. Thinking this through, more importantly, I should give audible feedback of the CPU opponent's move, otherwise how will the blind person know what counter-move the CPU has made? I also need to give this some thought. Perhaps run the game through the Command Line Interface rather than the GUI..

Thanks again.

Oct 11, 2016 • #3

I put together an example VoiceBot Profile that should work for you. When you actived the 'Move' macro in the profile, a script will run that asks you for a list of moves. Then you can say a list of moves, and it the script recognizes what you said, VoiceBot will type the command out for you, then let you know what it typed.

This script uses an external assembly file, called 'System.Speech.dll', and you will need to find that file on your computer to get this script to work correctly. I entered where it is on my computer, so the script might work, but there is a chance that it will be in a different location on your computer.

Here's how to get the script to work:

Download the .voicebot file attached to this post
Open the VoiceBot Settings window to the 'Profiles' tab
Click the 'Import' button, browse to the file you downloaded in the first step, then click OK
Open the Profile you just imported, Click on the 'Move' Macro, then click on the 'Macro Script' action
In the window that pops up with the code, click the 'Verify Function' button
If it verifies successfully, you're done! If not, continue to the next step
Replace the path to 'System.Speech.dll' to where it's located on your computer

I hope this works for you!

• Attachment: GNU Backgammon (English United States).voicebot [22,520 bytes]

• Attachment: ReferencedAssemblies.png [10,584 bytes]

ReferencedAssemblies.png

Oct 21, 2016 (modified Oct 21, 2016) • #4

djskope

11 discussion posts

Thank you very much for the macro Thomas and sorry for the long delay in response - I have taken a break from computer backgammon for some months.

I tried the profile today but even though the macro seemed to verify ok and also run ok, it did not seem to do anything at all.

I was having some trouble also with it recognising the digits 1 thru 10 as digits rather than words (one - ten) but this is a secondary problem.

I also changed the 'done' recogniser inside the script to the more distinctive word 'complete' as it was having trouble recognising 'done'.

Gnubg correctly receives normal spoken commands as text in its 'command box' for my basic commands, but no joy with your macro.

I have attached my amended profile which I copy / pasted your macro code into (at the bottom).

If I understood your instructions properly, then a sentence like 'move 24 20 24 18 done' should have worked? (or in my amended version of the macro 'move 24 20 24 18 complete' )

Perhaps I am doing something wrong?

Perhaps if you find some time in future you could download the freeware gnubg and try at your end?

http://gnubg.org/index.php?itemid=22

Interested to know your thoughts on this..

Thanks again

djskope

• Attachment: gnubg v1 (English United Kingdom).voicebot [33,430 bytes]

Feb 27, 2017 • #5

Hello again! One thing I forgot to mention in my last post, is that the "Command" text box will need to be visible and focused for this script to work. You can do this by opening up GNU Backgammon, then selecting View->Panels->Command. The script then asks you how you want to move, then types out the command for you.

Changing the last word to complete is a great idea! The speech recognition itself is something that you'll have to play around with because it is very finicky.

Hopefully this sets you on the right track!

Feb 27, 2017 • #6

djskope

11 discussion posts

Thanks for your quick response Thomas.

Yes, I certainly made sure the 'Command' box was in focus in GNUBG and the cursor was flashing there. I ran it side by side with Voicebot. Other simple commands from my custom GNUBG trigger fine in the same setup.

I'm not coder, but it seems strange that the macro does not even seem to reach the 1st step of user interaction for me..

Code
Copy
Select All

//ask the user where they would like to move
            BFS.Speech.TextToSpeech("Where would you like to move?");

i.e. this audible speech question does not happen on my machine.

Macro tests appear to verify and run correctly (background turns green).

May I ask, can you verify that your move macro works successfully with an installation GNUBG at your end? I feel maybe something simple is awry.

Thanks again,

djskope

Feb 28, 2017 • #7

djskope

11 discussion posts

Just a thought.. maybe I am not hearing the voice asking me for my move because I am not using the default voice?

Do I need to edit the original macro code code to account for this?

Feb 28, 2017 • #8

That's really strange, something must be going on behind the scenes here. I've got the script to work on my machine, and even enter text for me. Is it possible I could get a log from you, to see if the script or text to speech is throwing any errors? Here's how to do that:

On the Settings > Troubleshooting tab, change the Logging drop-down to "L1: Log Minimal"
Restart VoiceBot
Reproduce the issue and note the time so we'll know where to check in the log file
Send us the VoiceBot.log and DebugInfo.html files (can be found by clicking the Open Log button on the Troubleshooting tab)
Disable debug logging after sending the log

Thanks!

Feb 28, 2017 • #9

djskope

11 discussion posts

Thanks Thomas.

If I am reading the log correctly, I think the best example starts from 2017/02/28 20:03:10.035 containing the word 'move'.

You will see I made a lot of pronunciation mistakes in the full log - please ignore these. I need to retrain the Microsoft voice recognition engine as it does not reliably recognise all of my words.

I have attached the complete log for more context anyway.

>> VoiceBot.log and DebugInfo.html

VoiceBot.log is attached below but unfortunately I searched my C: drive but no file named DebugInfo.html existed??

So I have copy / pasted all the text from the troubleshooting info window (looks like my system info) into a text file and also attached (protected) - I hope it helps.

I appreciate your help - please let me know what you think whenever you have some time.

Thanks again,

djskope

Code
Copy
Select All

2017/02/28 20:03:10.035##_##INFO:L1##_##voicebot.exe:7292##_##BFScriptCodeRun:move:RunFunctionTHREAD##_##-##_##Function Start: move##_##-
2017/02/28 20:03:10.035##_##INFO:L1##_##voicebot.exe:7292##_##BFScriptCode:Run##_##-##_##Check: move##_##-
2017/02/28 20:03:10.073##_##INFO:L1##_##voicebot.exe:7292##_##FormMain:AddLogItem_UITHREAD##_##-##_##Text: move, macro: move, Actions: Macro Script##_##-
2017/02/28 20:03:10.075##_##INFO:L1##_##voicebot.exe:7292##_##BFNativeWindowBase:CTR##_##-##_##Creating on Thread: GUIThread:Main [Name: MessageOnly:BFTimer:Tray Icon Success • ID: 672d9a50-a6c8-4818-8386-79665b35a138]##_##-
2017/02/28 20:03:10.193##_##INFO:L1##_##voicebot.exe:7292##_##BFScriptCode:Run##_##-##_##Run: move##_##-
2017/02/28 20:03:10.209##_##FAIL:L1##_##voicebot.exe:7292##_##BFScriptCode:Run##_##-##_##Exception has been thrown by the target of an invocation. • The language for the grammar does not match the language of the speech recognizer.##_##-
2017/02/28 20:03:10.209##_##FAIL:L1##_##voicebot.exe:7292##_##BFScriptCodeRun:RunCodeTHREAD##_##-##_##Failed to run Function: Run Failed. • Exception has been thrown by the target of an invocation. • The language for the grammar does not match the language of the speech recognizer.##_##-
2017/02/28 20:03:11.581##_##INFO:L1##_##voicebot.exe:7292##_##BFNativeWindowBase:CloseWindow##_##-##_##Closing: BFDispose:DisposeTimer:BFTimer [Name: MessageOnly:BFTimer:Tray Icon Success • ID: 672d9a50-a6c8-4818-8386-79665b35a138]##_##-
2017/02/28 20:03:11.581##_##INFO:L1##_##voicebot.exe:7292##_##BFNativeWindowBase:DisposeWorkerPRIVATE##_##-##_##Disposing. [Name: MessageOnly:BFTimer:Tray Icon Success • ID: 672d9a50-a6c8-4818-8386-79665b35a138]##_##-
2017/02/28 20:03:12.187##_##INFO:L1##_##voicebot.exe:7292##_##FormMain:AddLogItem_UITHREAD##_##-##_##Text: 24, macro: -, Actions: -##_##-
2017/02/28 20:03:14.343##_##INFO:L1##_##voicebot.exe:7292##_##FormMain:AddLogItem_UITHREAD##_##-##_##Text: 23, macro: -, Actions: -##_##-
2017/02/28 20:03:16.527##_##INFO:L1##_##voicebot.exe:7292##_##FormMain:AddLogItem_UITHREAD##_##-##_##Text: 24, macro: -, Actions: -##_##-
2017/02/28 20:03:18.428##_##INFO:L1##_##voicebot.exe:7292##_##FormMain:AddLogItem_UITHREAD##_##-##_##Text: 21, macro: -, Actions: -##_##-
2017/02/28 20:03:19.946##_##INFO:L1##_##voicebot.exe:7292##_##FormMain:AddLogItem_UITHREAD##_##-##_##Text: complete, macro: -, Actions: -##_##-

• Attachment [protected]: troubleshooting information - djskope gnubg.txt [64,008 bytes]

• Attachment [protected]: VoiceBot.log [94,697 bytes]

Feb 28, 2017 • #10

I think I found the issue, and it looks like it's a problem with the script! The code I gave you loads the wrong speech recognizer. You can fix that by specifying which language the SpeechRecognitionEngine uses by replacing line 55:

Code
Copy
Select All

using(SpeechRecognitionEngine engine = new SpeechRecognitionEngine())

with this code:

Code
Copy
Select All

using(SpeechRecognitionEngine engine = new SpeechRecognitionEngine(new System.Globalization.CultureInfo("en-uk")))

I hope this works for you!

Mar 6, 2017 (modified Mar 6, 2017) • #11

djskope

11 discussion posts

Hi Thomas,

Thanks for solving this - I only had to make one minor change.

The SpeechRecognitionEngine didn't work with 'en-uk' on my machine. I changed it to 'en-us' and it immediately started working

Code
Copy
Select All

using(SpeechRecognitionEngine engine = new SpeechRecognitionEngine(new System.Globalization.CultureInfo("en-us")))

I have attached my updated profile (gnubg v2 (English United States).voicebot) - please feel free to share on your site.

Secondary to this, the 'move' command still gets a little bit confused with converting spoken numbers to digits (especially below 20).

i.e.
19 sometimes converts to '90',
18 converts to '80',
6 converts to 'six',
3 converts to 'three'.

Is there any way internally to restrict interpretation of spoken numbers to digits? Maybe an on/off toggle for 'force spoken numbers to digits / or words'?

And possibly a way to optimise / restrict interpreted range of the numbers from 13 to 19 (not 30 to 90)?

Thanks again!

djskope

• Attachment: gnubg v2 (English United States).voicebot [34,764 bytes]

Mar 7, 2017 (modified Mar 7, 2017) • #12

The only things I can suggest to get it working the way you want, is to make sure the "validMoves" collection on line 13 only contains the numbers you want, and to train your system to be more familiar with your voice.

Sorry I don't have a better answer, but I hope this helps!

Mar 21, 2017 • #13

djskope

11 discussion posts

Many thanks Thomas!

Mar 22, 2017 • #14

Was this helpful?

(1)

(-)