You are previewing Natural Language Generation in Interactive Systems.
O'Reilly logo
Natural Language Generation in Interactive Systems

Book Description

An informative and comprehensive overview of the state-of-the-art in natural language generation (NLG) for interactive systems, this guide serves to introduce graduate students and new researchers to the field of natural language processing and artificial intelligence, while inspiring them with ideas for future research. Detailing the techniques and challenges of NLG for interactive applications, it focuses on the research into systems that model collaborativity and uncertainty, are capable of being scaled incrementally, and can engage with the user effectively. A range of real-world case studies is also included. The book and the accompanying website feature a comprehensive bibliography, and refer the reader to corpora, data, software and other resources for pursuing research on natural language generation and interactive systems, including dialog systems, multimodal interfaces and assistive technologies. It is an ideal resource for students and researchers in computational linguistics, natural language processing and related fields.

Table of Contents

  1. Cover
  2. Half-title page
  3. Title
  4. Copyright
  5. Contents
  6. List of contributors
  7. 1 Introduction
    1. 1.1 Natural language generation
    2. 1.2 Interactive systems
    3. 1.3 Natural language generation for interactive systems
      1. 1.3.1 Collaboration
      2. 1.3.2 Reference
      3. 1.3.3 Handling uncertainty
      4. 1.3.4 Engagement
      5. 1.3.5 Evaluation and shared tasks
    4. 1.4 Summary
    5. References
  8. Part I Joint construction
    1. 2 Communicative intentions and natural language generation
      1. 2.1 Introduction
      2. 2.2 What are communicative intentions?
      3. 2.3 Communicative intentions in interactive systems
        1. 2.3.1 Fixed-task models
        2. 2.3.2 Plan-based models
        3. 2.3.3 Conversation Acts Theory
        4. 2.3.4 Rational behavior models
      4. 2.4 Modeling communicative intentions with problem solving
        1. 2.4.1 Collaborative problem solving
        2. 2.4.2 Collaborative problem solving state
        3. 2.4.3 Grounding
        4. 2.4.4 Communicative intentions
      5. 2.5 Implications of collaborative problem solving for NLG
      6. 2.6 Conclusions and future work
      7. References
    2. 3 Pursuing and demonstrating understanding in dialogue
      1. 3.1 Introduction
      2. 3.2 Background
        1. 3.2.1 Grounding behaviors
        2. 3.2.2 Grounding as a collaborative process
        3. 3.2.3 Grounding as problem solving
      3. 3.3 An NLG model for flexible grounding
        1. 3.3.1 Utterances and contributions
        2. 3.3.2 Modeling uncertainty in interpretation
        3. 3.3.3 Generating under uncertainty
        4. 3.3.4 Examples
      4. 3.4 Alternative approaches
        1. 3.4.1 Incremental common ground
        2. 3.4.2 Probabilistic inference
        3. 3.4.3 Correlating conversational success with grounding features
      5. 3.5 Future challenges
        1. 3.5.1 Explicit multimodal grounding
        2. 3.5.2 Implicit multimodal grounding
        3. 3.5.3 Grounding through task action
      6. 3.6 Conclusions
      7. References
    3. 4 Dialogue and compound contributions
      1. 4.1 Introduction
      2. 4.2 Compound contributions
        1. 4.2.1 Introduction
        2. 4.2.2 Data
        3. 4.2.3 Incremental interpretation vs. incremental representation
        4. 4.2.4 CCs and intentions
        5. 4.2.5 CCs and coordination
        6. 4.2.6 Implications for NLG
      3. 4.3 Previous work
        1. 4.3.1 Psycholinguistic research
        2. 4.3.2 Incrementality in NLG
        3. 4.3.3 Interleaving parsing and generation
        4. 4.3.4 Incremental NLG for dialogue
        5. 4.3.5 Computational and formal approaches
        6. 4.3.6 Summary
      4. 4.4 Dynamic Syntax (DS) and Type Theory with Records (TTR)
        1. 4.4.1 Dynamic Syntax
        2. 4.4.2 Meeting the criteria
      5. 4.5 Generating compound contributions
        1. 4.5.1 The DyLan dialogue system
        2. 4.5.2 Parsing and generation co-constructing a shared data structure
        3. 4.5.3 Speaker transition points
      6. 4.6 Conclusions and implications for NLG systems
      7. References
  9. Part II Reference
    1. 5 Referability
      1. 5.1 Introduction
      2. 5.2 An algorithm for generating boolean referring expressions
      3. 5.3 Adding proper names to REG
      4. 5.4 Knowledge representation
        1. 5.4.1 Relational descriptions
        2. 5.4.2 Knowledge representation and REG
        3. 5.4.3 Description Logic for REG
      5. 5.5 Referability
      6. 5.6 Why study highly expressive REG algorithms?
        1. 5.6.1 Sometimes the referent could not be identified before
        2. 5.6.2 Sometimes they generate simpler referring expressions
        3. 5.6.3 Simplicity is not everything
        4. 5.6.4 Complex content does not always require a complex form
        5. 5.6.5 Characterizing linguistic competence
      7. 5.7 Whither REG?
      8. References
    2. 6 Referring expression generation in interaction: A graph-based perspective
      1. 6.1 Introduction
        1. 6.1.1 Referring expression generation
        2. 6.1.2 Preferences versus adaptation in reference
      2. 6.2 Graph-based referring expression generation
        1. 6.2.1 Scene graphs
        2. 6.2.2 Referring graphs
        3. 6.2.3 Formalizing reference in terms of subgraph isomorphism
        4. 6.2.4 Cost functions
        5. 6.2.5 Algorithm
        6. 6.2.6 Discussion
      3. 6.3 Determining preferences and computing costs
      4. 6.4 Adaptation and interaction
        1. 6.4.1 Experiment I: adaptation and attribute selection
        2. 6.4.2 Experiment II: adaptation and overspecification
      5. 6.5 General discussion
      6. 6.6 Conclusion
      7. References
  10. Part III Handling uncertainty
    1. 7 Reinforcement learning approaches to natural language generation in interactive systems
      1. 7.1 Motivation
        1. 7.1.1 Background: Reinforcement learning approaches to NLG
        2. 7.1.2 Previous work in adaptive NLG
      2. 7.2 Adaptive information presentation
        1. 7.2.1 Corpus
        2. 7.2.2 User simulations for training NLG
        3. 7.2.3 Data-driven reward function
        4. 7.2.4 Reinforcement learning experiments
        5. 7.2.5 Results: Simulated users
        6. 7.2.6 Results: Real users
      3. 7.3 Adapting to unknown users in referring expression generation
        1. 7.3.1 Corpus
        2. 7.3.2 Dialogue manager and generation modules
        3. 7.3.3 Referring expression generation module
        4. 7.3.4 User simulations
        5. 7.3.5 Training the referring expression generation module
        6. 7.3.6 Evaluation with real users
      4. 7.4 Adaptive temporal referring expressions
        1. 7.4.1 Corpus
        2. 7.4.2 User simulation
        3. 7.4.3 Evaluation with real users
      5. 7.5 Research directions
      6. 7.6 Conclusions
      7. References
    2. 8 A joint learning approach for situated language generation
      1. 8.1 Introduction
      2. 8.2 GIVE
        1. 8.2.1 The GIVE-2 corpus
        2. 8.2.2 Natural language generation for GIVE
        3. 8.2.3 Data annotation and baseline NLG system
      3. 8.3 Hierarchical reinforcement learning for NLG
        1. 8.3.1 An example
        2. 8.3.2 Reinforcement learning with a flat state–action space
        3. 8.3.3 Reinforcement learning with a hierarchical state–action space
      4. 8.4 Hierarchical reinforcement learning for GIVE
        1. 8.4.1 Experimental setting
        2. 8.4.2 Experimental results
      5. 8.5 Hierarchical reinforcement learning and HMMs for GIVE
        1. 8.5.1 Hidden Markov models for surface realization
        2. 8.5.2 Retraining the learning agent
        3. 8.5.3 Results
      6. 8.6 Discussion
      7. 8.7 Conclusions and future work
      8. References
  11. Part IV Engagement
    1. 9 Data-driven methods for linguistic style control
      1. 9.1 Introduction
      2. 9.2 PERSONAGE: personality-dependent linguistic control
      3. 9.3 Learning to control a handcrafted generator from data
        1. 9.3.1 Overgenerate and rank
        2. 9.3.2 Parameter estimation models
      4. 9.4 Learning a generator from data using factored language models
      5. 9.5 Discussion and future challenges
      6. References
    2. 10 Integration of cultural factors into the behavioral models of virtual characters
      1. 10.1 Introduction
      2. 10.2 Culture and communicative behaviors
        1. 10.2.1 Levels of culture
        2. 10.2.2 Cultural dichotomies
        3. 10.2.3 Hofstede’s dimensional model and synthetic cultures
      3. 10.3 Levels of cultural adaptation
        1. 10.3.1 Culture-specific adaptation of context
        2. 10.3.2 Culture-specific adaptation of form
        3. 10.3.3 Culture-specific communication management
      4. 10.4 Approaches to culture-specific modeling for embodied virtual agents
        1. 10.4.1 Top-down approaches
        2. 10.4.2 Bottom-up approaches
      5. 10.5 A hybrid approach to integrating culture-specific behaviors into virtual agents
        1. 10.5.1 Cultural profiles for Germany and Japan
        2. 10.5.2 Behavioral expectations for Germany and Japan
        3. 10.5.3 Formalization of culture-specific behavioral differences
        4. 10.5.4 Computational models for culture-specific conversational behaviors
        5. 10.5.5 Simulation
        6. 10.5.6 Evaluation
      6. 10.6 Conclusions
      7. References
    3. 11 Natural language generation for augmentative and assistive technologies
      1. 11.1 Introduction
      2. 11.2 Background on augmentative and alternative communication
        1. 11.2.1 State of the art
        2. 11.2.2 Related research
        3. 11.2.3 Diversity in users of AAC
        4. 11.2.4 Other AAC challenges
      3. 11.3 Application areas of NLG in AAC
        1. 11.3.1 Helping AAC users communicate
        2. 11.3.2 Teaching communication skills to AAC users
        3. 11.3.3 Accessibility: Helping people with visual impairments access information
        4. 11.3.4 Summary
      4. 11.4 Example project: “How was School Today...?”
        1. 11.4.1 Use case
        2. 11.4.2 Example interaction
        3. 11.4.3 NLG in “How was School Today...?”
        4. 11.4.4 Current work on “How was School Today...?”
      5. 11.5 Challenges for NLG and AAC
        1. 11.5.1 Supporting social interaction
        2. 11.5.2 Narrative
        3. 11.5.3 User personalization
        4. 11.5.4 System evaluation
        5. 11.5.5 Interaction and dialogue
      6. 11.6 Conclusions
      7. References
  12. Part V Evaluation and shared tasks
    1. 12 Eye tracking for the online evaluation of prosody in speech synthesis
      1. 12.1 Introduction
      2. 12.2 Experiment
        1. 12.2.1 Design and materials
        2. 12.2.2 Participants and eye-tracking procedure
      3. 12.3 Results
      4. 12.4 Interim discussion
      5. 12.5 Offline ratings
        1. 12.5.1 Design and materials
        2. 12.5.2 Results
      6. 12.6 Acoustic analysis using Generalized Linear Mixed Models (GLMMs)
        1. 12.6.1 Acoustic factors and looks to the area of interest
        2. 12.6.2 Relationship between ratings and looks
        3. 12.6.3 Correlation between rating and acoustic factors
      7. 12.7 Discussion
      8. 12.8 Conclusions
      9. References
    2. 13 Comparative evaluation and shared tasks for NLG in interactive systems
      1. 13.1 Introduction
      2. 13.2 A categorization framework for evaluations of automatically generated language
        1. 13.2.1 Evaluation measures
        2. 13.2.2 Higher-level quality criteria
        3. 13.2.3 Evaluation frameworks
        4. 13.2.4 Concluding comments
      3. 13.3 An overview of evaluation and shared tasks in NLG
        1. 13.3.1 Component evaluation: Referring Expression Generation
        2. 13.3.2 Component evaluation: Surface Realization
        3. 13.3.3 End-to-end NLG systems: data-to-text generation
        4. 13.3.4 End-to-end NLG systems: text-to-text generation
        5. 13.3.5 Embedded NLG components
        6. 13.3.6 Embedded NLG components: the GIVE shared task
        7. 13.3.7 Concluding comments
      4. 13.4 An overview of evaluation for spoken dialogue systems
        1. 13.4.1 Introduction
        2. 13.4.2 Realism and control
        3. 13.4.3 Evaluation frameworks
        4. 13.4.4 Shared tasks
        5. 13.4.5 Discussion
        6. 13.4.6 Concluding comments
      5. 13.5 A methodology for comparative evaluation of NLG components in interactive systems
        1. 13.5.1 Evaluation model design
        2. 13.5.2 An evaluation model for comparative evaluation of NLG modules in interactive systems
        3. 13.5.3 Context-independent intrinsic output quality
        4. 13.5.4 Context-dependent intrinsic output quality
        5. 13.5.5 User satisfaction
        6. 13.5.6 Task effectiveness and efficiency
        7. 13.5.7 System purpose success
        8. 13.5.8 A proposal for a shared task on referring expression generation in dialogue context
        9. 13.5.9 GRUVE: A shared task on instruction giving in pedestrian navigation
        10. 13.5.10 Concluding comments
      6. 13.6 Conclusion
      7. References
  13. Author index
  14. Subject index