Creating Spoken Dialog Frameworks in the Communicator/RavenClaw System

0
0
2097 days ago, 760 views
PowerPoint PPT Presentation
SPHINX. Amalgamation. THETA. Universe. Bland concentrated, message-passing correspondence ... SPHINX. SPHINX. SPHINX. Getting Even Closer. Dialog Manag. RAVENCLAW. Back ...

Presentation Transcript

Slide 1

Creating Spoken Dialog Systems in the Communicator/RavenClaw Framework Sphinx Lunch Talk Carnegie Mellon University, October 2004 Presented by: Dan Bohus Special appearances: Antoine Raux, Jahanzeb Sherwani, Thomas Harris

Slide 2

Examples RoomLine gathering room reservations inside SCS; framework can get to calendars of 13 conf rooms in Wean-Hall and NSH Let's Go! Transport Information System transport plan data framework for Port Authority transports in Oakland and Squirrel Hill [Let's Go! Project] Sublime customized data administration framework TeamTalk an examination concerning human and multi-robot talked dialect correspondence in unstructured situations

Slide 3

Examples RoomLine gathering room reservations inside SCS; framework can get to timetables of 13 conf rooms in Wean-Hall and NSH Let's Go! Transport Information System transport plan data framework for Port Authority transports in Oakland and Squirrel Hill [Let's Go! Project] Sublime customized data administration framework TeamTalk an examination concerning human and multi-robot talked dialect correspondence in unstructured situations

Slide 4

Examples RoomLine meeting room reservations inside SCS; framework can get to calendars of 13 conf rooms in Wean-Hall and NSH Let's Go! Transport Information System transport plan data framework for Port Authority transports in Oakland and Squirrel Hill [Let's Go! Project] Sublime customized data administration framework TeamTalk an examination concerning human and multi-robot talked dialect correspondence in unstructured situations

Slide 5

Examples RoomLine meeting room reservations inside SCS; framework can get to timetables of 13 conf rooms in Wean-Hall and NSH Let's Go! Transport Information System transport plan data framework for Port Authority transports in Oakland and Squirrel Hill [Let's Go! Project] Sublime customized data administration framework TeamTalk an examination concerning human and multi-robot talked dialect correspondence in unstructured situations

Slide 6

More Systems LARRI multimodal framework that helps F/A-18 flying machine upkeep faculty all through the execution of procedural errands [Symphony] Madeleine content based model for medicinal determination framework [MITRE workshop] Eureka discourse interface to the Vivisimo web search tool

Slide 7

The Communicator/RavenClaw Spoken Dialog Systems Framework Examples Overall Architecture System Development Components & Resources Miscellaneous Current Research cases : engineering : improvement : parts : various : inquire about

Slide 8

Recognition SPHINX Synthesis THETA Overall Architecture Classical pipeline design Lang. Get it. PHOENIX/HELIOS Dialog Manag. RAVENCLAW Back-end (different) Lang. Era ROSETTA illustrations : engineering : improvement : segments : different : examine

Slide 9

Galaxy HUB Generic concentrated, message-passing correspondence design Developed at MIT, utilized as a part of Communicator program Competitor: OAA Recognition SPHINX Lang. Get it. PHOENIX/HELIOS Galaxy HUB Dialog Manag. RAVENCLAW Back-end (different) Synthesis THETA Lang. Era ROSETTA cases : design : improvement : segments : different : look into

Slide 10

Getting Even Closer Recognition SPHINX Lang. Get it. PHOENIX/HELIOS HUB Dialog Manag. RAVENCLAW Back-end (perl) Synthesis THETA Language Gen. ROSETTA illustrations : engineering : advancement : segments : various : look into

Slide 11

Inputs from different modalities Other area specialists DateTime Parsing PHOENIX Lang. Get it. PHOENIX/HELIOS Confidence HELIOS Back-end Galaxy Stub Lang. Era Galaxy Stub Actual Perl Back-end Lang. Era ROSETTA (Perl) Text I/O TTYServer PROCESS MONITOR Getting Even Closer Multiple, parallel decoders SPHINX Recognition Server HUB Dialog Manag. RAVENCLAW Back-end (perl) Synthesis THETA Lang. Era ROSETTA illustrations : engineering : advancement : segments : various : investigate

Slide 12

The Communicator/RavenClaw Spoken Dialog Systems Framework Examples Overall Architecture System Development Components & Resources Miscellaneous cases : design : improvement : parts : random : look into

Slide 13

Recognition SPHINX Synthesis THETA Building a Spoken Dialog System Language, Acoustic, Lexical Models Grammar Lang. Get it. PHOENIX/HELIOS Dialog Manag. RAVENCLAW Back-end (perl) Back-end (perl) RavenClaw Dialog Task Specification Lang. Era ROSETTA (Limited Domain) Voice Templates cases : engineering : improvement : parts : different : investigate

Slide 14

Recognition SPHINX Synthesis THETA So How Long Will It Take? Miter Workshop on Dialog Management (Fall 2003) Develop a Text-based SDS for therapeutic analysis (gave backend) Madeleine (22 hours) Language, Acoustic, Lexical Models Grammar Lang. Get it. PHOENIX/HELIOS Dialog Manag. RAVENCLAW Back-end (perl) Back-end (perl) RavenClaw Dialog Task Specification Lang. Era ROSETTA (Limited Domain) Voice Templates cases : design : advancement : segments : incidental : investigate

Slide 15

Okay, How Long Will It Really Take? To get a framework running with a sensible execution [poll among 3 RavenClaw developers] 1 month to get a working framework up and running 1 month to adjust execution Further iterative enhancements will proceed as more information gathers cases : engineering : improvement : parts : different : look into

Slide 16

The Communicator/RavenClaw Spoken Dialog Systems Framework Examples Overall Architecture System Development Components & Resources Miscellaneous cases : design : advancement : segments : random : inquire about

Slide 17

Recognition SPHINX Synthesis THETA Components & Resources Language, Acoustic Models Grammar Lang. Get it. PHOENIX/HELIOS Dialog Manag. RAVENCLAW Back-end (perl) Back-end (perl) RavenClaw Dialog Task Specification Lang. Era ROSETTA Limited Domain Voice Templates cases : engineering : advancement : segments : various : look into

Slide 18

Components & Resources Language, Acoustic Models Grammar Recognition SPHINX Lang. Get it. PHOENIX/HELIOS Dialog Manag. RAVENCLAW Back-end (perl) Back-end (perl) RavenClaw Dialog Task Specification Synthesis THETA Lang. Era ROSETTA Limited Domain Voice Templates cases : design : advancement : parts : random : inquire about

Slide 19

SPHINX II Semi-consistent acoustic models Off-the-rack 8kHz, 11.025kHz, 16kHz models Scripts for building your own PLSA adjusted models perform better Language models 2-gram & 3-gram display CMU-Cambridge SLM Toolkit Generate from Phoenix Grammar Finite state sentence structure Sphinx underpins state-particular LMs Dictionary (lexical models) CMU Dictionary cases : engineering : improvement : segments : various : explore

Slide 20

Sphinx II - proceeded with Multiple parallel decoders [e.g., male + female] Multiple theory sent, choice done later Typical WER: 15-30% With proclaimed contrasts local versus non-local Lowered by retuning acoustic and dialect models to the space Migration to SPHINX 3.x sooner rather than later Expected: huge change in WER Concern: continuous execution

Slide 21

Recognition SPHINX Synthesis THETA Components & Resources Language, Acoustic Models Grammar Lang. Get it. PHOENIX/HELIOS Dialog Manag. RAVENCLAW Back-end (perl) Back-end (perl) RavenClaw Dialog Task Specification Lang. Era ROSETTA Limited Domain Voice Templates illustrations : engineering : advancement : parts : incidental : inquire about

Slide 22

Phoenix Parser/Grammar Phoenix: Robust Parser CFG Grammar Manually-created area particular linguistic use rules Reusable, non specific sub-language structures [Yes], [No], [Number], [DateTime], [Help], [Repeat], [Suspend], and so forth… [room_size_spec] ([rss_large]) ([rss_small]) ([rss_larger]) ([rss_smaller]) ([rss_smallest]) ([rss_largest]) ; [rss_large] (vast) (huge) (enormous) ; [rss_larger] (*the bigger) (*the greater) (too little) ; [rss_largest] (*the biggest) (*the greatest) ; [rss_small] (little) (little) ; DO YOU HAVE SOMETHING A BIT LARGER ? [NeedRoom] ( [_i_want] (DO YOU HAVE SOMETHING) ) [RoomSizeSpec] ( [room_size_spec] ( [rss_larger] (LARGER))) Parses all approaching theories and passes all parses along… illustrations : design : improvement : segments : different : look into

Slide 23

Helios/Confidence Annotation Builds precise certainty scores utilizing highlights from 3 wellsprings of learning: Speech acknowledgment Language understanding Dialog administration Selects speculation with most extreme certainty score Research in advance on theory choice, and transferability crosswise over spaces cases : engineering : advancement : parts : various : investigate

Slide 24

Recognition SPHINX Synthesis THETA Components & Resources Language, Acoustic Models Grammar Lang. Get it. PHOENIX/HELIOS Dialog Manag. RAVENCLAW Back-end (perl) Back-end (perl) RavenClaw Dialog Task Specification Lang. Era ROSETTA Limited Domain Voice Templates cases : design : advancement : segments : different : look into

Slide 25

RavenClaw Architecture Captures all area particular exchange (errand) rationale utilizing a progressive portrayal The writing exertion is centered around here Dialog Task (Specification) Domain-autonomous Dialog Engine Manages discourse by executing the exchange assignment determination Provides an expansive number of space free conversational systems illustrations : engineering : improvement : segments : various : inquire about

Slide 26

RavenClaw Architecture Captures all space particular discourse (undertaking) rationale with a various leveled depiction The creating exertion is centered around here Dialog Task (Specificati

SPONSORS