Talking to your computer!
Learn how to speak to your computer with Microsoft Speech Recognition along with a character to talk to with Microsoft Agent!
Anyone can talk to their computer! However, it just sits there like a rock not knowing you even said anything. If it does, then it sure as heck wouldn't understand you!
Microsoft has solved this problem. With one simple ActiveX control, we have the ability to talk to our computer. And with another simple control, we can add a professional character too!
- First, we need to download the Speech Recognition Engine (6 MB)
- Download and install it using the link above
- Click HERE for the download site. Click on the
Speech Control Panel button and download it.
- After it is installed, click on the first link in the index. Download and install it.
- Click on the link below it, and choose a character from the dropdown list. I like Robby the robot the most. There is also a bird (Peddy), and Genie, and Merlin.
You can use the others later. If you do not choose Robby, things will
not make sense.
- Then click on the download button to download the character. Install it, then continue with the next step.
- Now download the next two links, and install those. That will allow the computer to speak back to us. You should have clicked on every link but one.
So far, that should be easy. If you could not follow it, we clicked on every link in the contents at the top of the page except the bottom one. It may seem like we are ready, but we really aren't...
Training
If you try to speak to the computer now, it will have a very hard time "understanding" you. So, we need to train it.
- Enter the Control Panel. It can be found in the Start Menu.
- Find the Speech icon, and click on it. A dialog will open.
- Click on the "Other" tab at the top, the second one.
- Click on the "SAPI 4 Control Panel..." button to open yet another dialog.
- Click on the text in the list that reads "Microsoft Speech Recognition Engine 4.0 (English)" at the bottom.
- Click on the "Training..." button. Another dialog will open.
- NOTE: You may have done this already during the setup. It is wise to do it again, though.
- Choose a session from the list. None are too exciting...
- Click on the 'Next >' button to take you to the next screen.
- Get your microphone ready, because once you click on 'Next >' again, you will have to speak!
- It will ask you to read the story. Do NOT read abnormally! Read like you normally do so it learns how you speak.
- When you are done, it will update the computer. Keep doing it until you seem
comfortable. The more times you do it, the better it will get to know you!
Once that is done, you computer will better understand you. It takes a lot of work to get it good. Mine gets it correct about 90% of the time, but I still need work.
Bringing It Together
Now we are finally ready to start making our program. As I sit here typing this, I wonder exactly what this will do. Well, I now know, and you will know soon enough. Just follow all of the steps to get the final working program and be amazed at what Microsoft can do for us...
- Open Visual Basic and start a new Standard EXE project.
- Rename the form 'frmMain' and delete its caption property. The form will never be visible, so we do not need to worry about this.
- Set its Visible property to 'False' so we can not see it. We only want to see our little Agent.
- We now need to add the ActiveX controls to our project.
- Right-click on the toolbox and select 'Components'
- In the list, find and
select Microsoft Agent Control 2.0 and Microsoft Direct Speech
Recognition
- You will see an ear and an agent appear in your toolbox.
We now have everything we need added to our project. We can now start to
make it happen! We now must create a list of our commands to be loaded from a
text file.
Grammar
Open up Notepad. We must create our own file to load
the grammar and vocabulary from. It will only recognize what we enter in this
document. So, enter the following:
[Grammer]
type=cfg
[]
=Show Yourself
=Hide Yourself
=Speak
=Execute Notepad
=Execute Microsoft Paint
=Execute Microsoft Visual Basic
=Display Character Map
=Disallow Commands
=Reallow Commands
=Hibernate
=Make Repairs
=Tell Me a Joke
=What Is My Fortune
=Play Number Guessing Game
=Status Report
=0
=1
=2
=3
=4
=5
=6
=7
=8
=9
=10
=Animate
=Display Animation List
=Display System Information
=System Shutdown
=System Reboot
Save it in the same
directory as the project, naming it 'words.txt'. Now that our grammar list has
been created, we can load it into our program. We first need to place the
ActiveX controls onto our form.
ActiveX Controls
Drag the two new
controls onto the form (the ones that look like an agent and an ear). Name the
ear-like one 'Reco', for it will be our Speech Recognition Engine. Then, set its
'Visible' property to 'False'. Next, nam
' e the agent 'Agent', so all you have to
'
do is erase the one. This does not have a visible property.
Our
Agent
Before we load the
list we just created, let's create our Agent object. You do not have to do it in
this order, but I just want you to see what our character will look like. Create
a new module and name it 'modMain'. Enter the following code in the General
Declarations section (at the top):
Public Bot As
IAgentCtlCharacterEx
This creates a new Agent character from an external source. Therefore, we
will have to load it. Double click on frmMain and enter the following code:
Agent.Characters.Load "Robby",
"robby.acs"
Set Bot = Agent.Characters("Robby")
Bot.Show
Bot.Play "Wave"
Bot.Speak "Hello!"
Bot.Speak "How are you doing today?"
Bot.MoveTo 200, 200
That is it! If you run the program now, you will see your robot wave to
you, speak, then move. Let's take a look at the code.
The first two lines load the bot. The 'robby.acs' is the file we load it
from. The third line shows our bot. The Bot.Play "Wave" line will play his
'Wave' animation. We will take a look at
' animating much more in the future.
Then, we tell the bot to speak with Bot.Speak. Finally, the MoveTo command will
move the bot to a different part of the screen.
Before you move on, try to do some different things with it. Modify what
it says when it appears, move it to a different position on the screen.
Loading the Grammer
We can not speak to the computer until we load the list we
just created. Enter the following code above all the code we just put in the
Form_Load procedure:
Reco.GrammarFromFile App.Path & "\Words.txt"
Reco.Activate
To see if it loaded correctly, we will make a simple example. If you speak
a phrase we created, it will output it to the debug window. Please note that it
may not recognize you every time. Only speak what is in our list. Double-click
on Reco and enter this in its Phrase_Finish procedure:
Debug.Print Phrase
As you speak, look in the Debug Window. If it outputs any of the phrases
listed in Words.txt, we can continue.
Responding
How do we respond to this? Well, we need to replace the
one line of code we just entered with a little more:
On Local Error Resume Next
Debug.Print Phrase
Select Case Phrase
Case "Show Yourself"
Bot.Show
Case "Hide Yourself"
Bot.Hide
End Select
Please note that it is case sensitive, so you must type it
exactly as it appears above. All we do is output the phrase said to the
Debug Window, then check it. These two phrases are the basic ones, they show and
hide the bot. If it is 'Show Yourself', then the bot will be displayed. If it is
'Hide Yourself', we will hide the bot. I
' t is that simple. Try it out and see if
'
it works for you. Please be patient, it may take a while to recognize...
Speaking
We will now program
our bot to speak to us. Please modify the above code so it looks like this:
On
Local Error Resume Next
Debug.Print Phrase
Select Case Phrase
Case "Show Yourself"
Bot.Show
Case "Hide Yourself"
Bot.Hide
Case "Speak"
BotSpeak
End Select
In the above code, we
only added two lines. It will call a sub procedure which will randomly choose
something for our bot to say. In modMain, create a new sub procedure named
'BotSpeak':
Public Sub BotSpeak()
Randomize
Dim r As
Integer
r = Int(16 * Rnd)
Select Case r
Case 0
Bot.Speak "I
love you!"
Case 1
Bot.Speak
"You are amazing!"
Case 2
Bot.Speak
"Make me a sandwich!"
Case 3
Bot.Speak
"I'm tired..."
Case 4
Bot.Speak
"Hello!"
Case 5
Bot.Speak
"Dreams do come true!"
Case 6
Bot.Speak
"Solve the equation 8(2x + 3) - (5x + 3x + 4)"
Case 7
Bot.Speak
"What is the meaning of life?"
Case 8
Bot.Speak "I
hate you, go away!"
Case 9
Bot.Speak
"Make me!"
Case 10
Bot.Speak
"I'm so happy being with you!"
Case 11
Bot.Speak
"Have you read the book 844454XC50?"
Case 12
Bot.Speak "I
want some self time..."
Case 13
Bot.Speak
"Humans are obsolete!"
Case 14
Bot.Speak "I
can not wait to take over your world!"
Case 15
Bot.Speak
"South Park rules!"
End Select
End Sub
You should be able to
understand that. We first choose a random number. Then, we tell the bot to speak
the text accociated to that number. That is it! Run the program, say "Speak",
and see what it says. Feel free to change the text to whatever you'd like.
Executing Applications
This is a very easy
topic to cover. All you have to do is add the following code to what we
currently have in the Phrase_Finish procedure:
Case "Execute Notepad"
Shell "C:\Windows\NOTEPAD.exe", vbNormalFocus
Case "Execute Microsoft Paint"
Shell "C:\WINDOWS\SYSTEM32\mspaint.exe", vbNormalFocus
Case "Execute Microsoft Visual Basic"
Shell "C:\Program Files\Microsoft Visual
Studio\VB98\VB6.exe", vbNormalFocus
Case "Display Character Map"
Shell "C:\WINDOWS\SYSTEM32\charmap.exe", vbNormalFocus
The 'Shell' command will execute an EXE file. The first parmeter is the
Path, where the executable is located. The second is the focus. Do you want it
to have the focus, do you want it to be minimized, maximized, or normal?
Disallow and Reallow
Before we get started, add a public Boolean variable named
'Disallow'. Now, add the following code
' at the top of the Phrase_Finish
procedure:
If Phrase = "Reallow Commands"
And Disallow Then
Bot.Play "DoMagic1"
Bot.Play "DoMagic2"
Bot.Play "Idle1_1"
Disallow = False
Bot.Speak "Welcome back!"
End If
If Phrase = "Disallow Commands"
Then
Bot.Play "DoMagic1"
Bot.Play "DoMagic2"
Disallow = True
Bot.Play "Idle1_1"
Bot.Speak "I will wait for your return!"
Exit Sub
End If
If Disallow
Then Exit Sub
Run the program and say 'Disallow Commands'. Now, try to get
it to speak. You can't. Now say 'Reallow Commands'. It can speak again! This is
good if you don't want someone messing around with your bot.
Hibernating and Making Repairs
These are only two animations. Hibernating will, of course,
but it to sleep. Make Repairs will make him take off his head, and make repairs.
Add the following code:
Case "Make Repairs"
Bot.Play "Idle2_2"
Case "Hibernate"
Bot.Play "Idle3_2"
You should understand that tiny amount of code. These two animations can
also be played if you leave your bot alone long enough.
Jokes
This is the same concept as speaking. You can just copy the
BotSpeak procedure and replace the lines with jokes. I will give you two. You
can come up with some of your own. Enter the following code in the Phrase_Finish
procedure:
Case "Tell Me a Joke"
TellJoke
Now, create the sub procedure in modMain:
Public Sub
TellJoke()
Dim r As
Integer
r = Int(2 * Rnd)
Bot.Play "GetAttention"
Select Case r
Case 0
Bot.Speak
"Why didn't the chicken cross the road?"
Bot.Speak
"Because he was chicken!"
Case 1
Bot.Speak
"Why didn't the man cross the road?"
Bot.Speak
"Because he didn't want to!"
Bot.Play
"Confused"
End Select
Bot.Play "GetAttentionReturn"
End Sub
Say "Tell Me a Joke" and listen. If you have any good jokes
that are yours (I have millions of them, but they are not mine. That is
why they are not on here), feel free to
send them to me!
Fortune Telling
Again, this is very similar to joke telling and speaking.
Enter the following lines in the Phrase_Finish procedure:
Case "What Is My Fortune"
TellFortune
Now, create the procedure in modMain:
Public Sub TellFortune()
Dim r As
Integer
r = Int(14 * Rnd)
Bot.Play "Process"
Select Case r
Case 0
Bot.Speak "Do
not go outside tomorrow..."
Case 1
Bot.Speak
"Keep looking behind your back!"
Case 2
Bot.Speak "In
the sky is your answer!"
Case 3
Bot.Speak
"ERROR - DEATH TOO SOON!"
Case 4
Bot.Speak
"Hold on to your belongings..."
Case 5
Bot.Speak "I
see great things in your future!"
Case 6
Bot.Speak "Do
NOT answer the door!"
Case 7
Bot.Speak
"tomorrow will be a great day for you!"
Case 8
Bot.Speak "It
says - 'NEXT BIRTHDAY'"
Case 9
Bot.Speak
"I'd rather not say!"
Bot.MoveTo
10, 10
Case 10
Bot.Speak
"Take shelter!"
Bot.Hide
Case 11
Bot.Speak
"You will be married tomorrow!"
Case 12
Bot.Speak
"You are dead, I'm afraid..."
Bot.Think "Ha
ha ha ha ha!"
Case 13
Bot.Speak
"ERROR - MISPRINT"
End Select
End Sub
There is something new up there. Bot.Think. Instead of speaking, the agent
displays a cloud with the text in it. The rest we have already gone through.
Number Guessing Game
This is a tiny bit more involved. First create a Boolean
variable in modMain called InGame, and an Integer variable named GuessNum. Now,
put this at the top of the Phrase_Finish sub:
If InGame
Then
GameInput Phrase
Exit Sub
End If
Now, add this to the same procedure:
Case "Play Number Guessing Game"
PlayGame
We have two procedures to create. Let's start with PlayGame:
Public Sub PlayGame()
GuessNum = Int(11 * Rnd)
Bot.Play "Explain"
Bot.Speak "You have three tries to guess the number I am
thinking of."
Bot.Speak "I will tell you if you are too high or too low..."
Bot.Play "Process"
Bot.Speak "I am thinking of a number between 0 and 10..."
InGame = True
End Sub
This procedure explains what you are to do, and chooses a
number. It then sets the InGame variable to 'True' allowing us to bypass all
other commands and worry only about the numbers we are guessing:
Public Sub GameInput(Phrase As String)
If Phrase = "0"
Or Phrase = "1" Or
Phrase = "2" Or Phrase = "3"
Or Phrase = "4" Or Phrase = "5"
Or Phrase = "6" Or Phrase = "7"
Or Phrase = "8" Or
Phrase = "9" Or Phrase = "10"
Then
Dim
tmpNum As Integer
Static
GNum As Integer
If GNum
= 3 Then GNum = 0
tmpNum = Int(Phrase)
If
tmpNum > GuessNum Then
Bot.Speak
CStr(tmpNum) & " - Number too high"
ElseIf
tmpNum < GuessNum Then
Bot.Speak
CStr(tmpNum) & " - Number too low"
ElseIf
tmpNum = GuessNum Then
Bot.Play
"Congratulate"
Bot.Speak
CStr(tmpNum) & " - That is correct!"
If GNum = 0 Then
Bot.Speak "WOW! First Try!"
InGame =
False
Exit Sub
End If
GNum = GNum + 1
If GNum
= 3 Then
Bot.Play
"Sad"
Bot.Speak
"You did not get it..."
Bot.Speak
"The correct number was " & CStr(GuessNum) & "..."
InGame =
False
End If
End If
End Sub
This is really some simple code. I simplified the first line
so it is easier to understand. We check to see if what the user said is actually
a number. If it is, we continue. If not, we bypass it. Then, we reset the tries
count (GNum). Then, we change the string into an integer and store it in tmpNum.
The bot will then inform us if it is too high or too low. If we are correct or
take our three turns, the game is over.
Status Report
This is nothing good. I just added this because every robot
must have status reports. It is also here if you decide to turn your robot into
a pet like a tamagotchi (I did). Every time, the bot will say he is operating a
optimum efficiency:
Case "Status Report"
Bot.Play "Explain"
Bot.Speak "I am operating at optimum efficiency!"
Animation
If you've looked at Microsoft Word or any of the Office
products, you can animate the agents. Yes, they are agents, so keep this
in mind: YOU CAN USE THEM IN YOUR APPS!
First, we are going to create the Animation List. Follow
these simple steps:
-
Create a new form and name it frmAni
-
Add a ListBox control to it, naming it lstAni
-
Add three CommandButtons: cmdPlay, cmdStop, and cmdCancel
-
In cmdStop, add this line of code: Bot.Stop
-
In cmdCancel, add this line: Unload Me
There, now we have the form set up. Make sure Option Explicit
is not at the top of the code for the form, then double-click on it and add the
following code to the Form_Load procedure:
For Each AnimationName
In Bot.AnimationNames
lstAni.AddItem AnimationName
Next
This will loop through all the animations and add them to our
list. You will see this soon. Add this to cmdPlay's code:
On Error GoTo
ErrOut
Bot.Stop
Bot.Play lstAni.Text
Exit Sub
ErrOut:
Bot.Speak "ERROR - UNABLE TO PLAY"
This will play the currently selected animation in the list. If there is
an error, our bot will tell us.
Now, we need to add the code for 'Auto Animation' when the user says
'Animate'. It will choose one from the l
' ist, play it, and never show the form.
Add a public Boolean variable named SelfAni in the General Declarations section
of the form, and add the following code below the other in the Form_Load
procedure:
If SelfAni Then
Dim r As
Integer
r = Int(lstAni.ListCount * Rnd)
lstAni.ListIndex = r
Bot.Stop
Bot.Play lstAni.Text
Unload Me
End If
If SelfAni is true, then it will not display the form and choose a random
animation. You will see how we will do this momentarily, so add the following
code in the Phrase_Finish procedure:
Case
"Display Animation List"
frmAni.SelfAni =
False
frmAni.Show
Case "Animate"
frmAni.SelfAni = True
frmAni.Show
Run the program and test out our two new commands. What do you think?
System Information
Another very easy topic. Enter the following code into the
Phrase_Finish procedure:
Case "Display System Information"
Shell "C:\Program Files\Common Files\Microsoft
Shared\MSINFO\msinfo32.exe", vbNormalFocus
This program will display your systems information. Our bot
has to do something that has to do with our computer.
Shut Down
These are the two last commands. By now, my wrists are
getting pretty sore from typing all of this! Add the following code above
everything in modMain:
Declare Function ExitWindowsEx Lib
"user32" (ByVal uFlags As
Long, ByVal dwReserved
As Long) As Boolean
Public Const EWX_LOGOFF = 0
Public Const EWX_REBOOT = 2
Public Const EWX_SHUTDOWN = 1
Now, we are almost done. As you know, we are only doing
Reboot and Shutdown. You can program Logoff, which is pathetically simple (as
you will soon see). Add the following code to finish this off:
Case "System Reboot"
Dim reboot
As Variant
reboot = MsgBox("Are you sure you want to reboot?", vbYesNo,
"Reboot?")
If reboot = vbYes
Then
Screen.MousePointer = vbHourglass
ExitWindowsEx EWX_REBOOT, 0
Unload Me
End
End If
Case "System Shutdown"
Dim down
As Variant
down = MsgBox("Are you sure you want to shutdown?", vbYesNo,
"Shutdown?")
If down = vbYes
Then
Screen.MousePointer = vbHourglass
ExitWindowsEx EWX_SHUTDOWN, 0
Unload Me
End
End If
Tips
Well, that is it! We are done. But before I go, here are some
tips and notes:
-
You can use Office's agents, and office can use the ones
you downloaded
-
I just scrapped the surface, there is much more to learn.
-
Add another agent, you can use more than one!
-
If you have any questions,
E-mail them to me!
-
Please Vote!!!