Voice Interaction

From KameRider
Jump to: navigation, search
  • to allow robot to understand human language, a software speech analyser is needed.
  • in this case, pocketphinx is our speech analyser.
  • this package will identify what the user speaks and then translate it into text. However, what the user speaks must be restricted according to the corpus/text library.
  • the topic publish by this package is /recognizer/output and this topic has message type of std_msgs/String which is String type message.
  • the message type can be referred at here: [1]
  • then, after robot can identify what the user spoke, robot can also speak back accordingly.
  • this can be done using package pi_speech_tutorial.
  • by initiate suitable sound that user prefer, robot can speaks a sentence that the user programmed.
  • to do this, user need to subscribe the /recognizer/output topic and extract the information inside the topic.
  • An example of source code with explanation:

include section:

1 # Include pi_speech_tutorial library
2 # This library has talkback function which user can implement
3 import roslib; roslib.load_manifest('pi_speech_tutorial')
4 
5 # Include function that can generate voice
6 from sound_play.libsoundplay import SoundClient
7 
8 # Include String type messsage
9 from std_msgs.msg import String

initiation section:

 1 # Identify the location of voice library and which voice that user prefer
 2 self.voice = rospy.get_param("~voice", "voice_don_diphone")
 3 
 4 # Identify the location of wave (a type of melody) library
 5 self.wavepath = rospy.get_param("~wavepath", "")
 6 
 7 # Create the sound client object which incharge in generatig voice
 8 self.soundhandle = SoundClient()
 9 
10 # Stop all function during initiation
11 self.soundhandle.stopAll()
12 
13 # Play a melody from wave library with named "R2D2.wav"
14 self.soundhandle.playWave(self.wavepath + "/R2D2a.wav")
15 
16 # Stop for 1 second
17 rospy.sleep(1)
18 
19 # Produce voice which speak "ready"
20 self.soundhandle.say("Ready", self.voice)
21 
22 # Subscribe to the /recognizer/output topic (from pocketphinx).
23 # What user spoke before will be convert into test/String format in this topic
24 rospy.Subscriber('/recognizer/output', String, self.identify)

callback function section:

 1 def identify(self, msg):
 2     # Print the recognized words on the screen then reply
 3     if msg.data == 'hello':
 4         rospy.loginfo(msg.data)
 5         self.soundhandle.say("hello", self.voice)
 6         # 1 second is given to the sound client to complete the sentence
 7         rospy.sleep(1)
 8     elif msg.data == 'how are you'
 9         rospy.loginfo(msg.data)
10         self.soundhandle.say("I am fine", self.voice)
11         rospy.sleep(1)
12     elif msg.data == 'who are you'
13         rospy.loginfo(msg.data)
14         self.soundhandle.say("I am turtle", self.voice)
15         rospy.sleep(1)

main (loop) section:

1     # USER CAN DO ANYTHING HERE
2     # IN THIS CASE, MAIN DO NOTHING
3 
4     # Sentence below makes the main loops at 5Hz
5     rospy.Rate(5).sleep()
  • source code of the above example can be found here: [2]
  • source code of integration of voice command with navigation can be found here: [3]