Abstract
Knowledge-aware dialogue generation aims to generate informative and meaningful responses with external knowledge. Existing works are still insufficient to encode retrieved knowledge regardless of the dialogue context, which probably leads to the introduction of irrelevant information. In this paper, we propose a dialogue generation model named CKFS-DG, which filters out context-irrelevant and off-topic knowledge to reduce the influence of redundant knowledge. Specifically, we design a knowledge-enriched encoder and a topic fact predictor to improve the quality of fusion knowledge. For achieve the knowledge-enriched encoder, we put forward a context-knowledge attention mechanism to dynamically filter out irrelevant knowledge conditioned on context. For the topic fact predictor, we utilize the probability distribution on retrieved facts to retain on-topic knowledge. The experimental results on English Reddit and Chinese Weibo dataset demonstrate that CKFS-DG outperforms the state-of-the-art neural generative methods in knowledge utilization, and CKFS-DG could reduce the influence of irrelevant knowledge to generate reasonable responses.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
Available at: https://conceptnet.io.
- 2.
References
Zhang, H., Liu, Z., Xiong, C., Liu, Z.: Grounded conversation generation as guided traverses in commonsense knowledge graphs. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 2031–2043 (2020)
Zhou, H., Young, T., Huang, M., Zhao, H., Xu, J., Zhu, X.: Commonsense knowledge aware conversation generation with graph attention. In: Proceedings of the 27th International Joint Conference on Artificial Intelligence, pp. 4623–4629 (2018)
Sutskever, I., Vinyals, O., Le, Q. V.: Sequence to Sequence Learning with Neural Networks. In: Advances in Neural Information Processing Systems, pp. 3104–3112 (2014)
Zhou, S., Rong, W., Zhang, J., Wang, Y., Shi, L., Xiong, Z.: Topic-aware dialogue generation with two-hop based graph attention. In: IEEE International Conference on Acoustics, Speech and Signal Processing, pp. 7428–7432 (2021)
Wu, S., Li, Y., Zhang, D., Zhou, Y., Wu, Z.: Diverse and informative dialogue generation with context-specific commonsense knowledge awareness. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 5811–5820 (2020)
Ghazvininejad, M., et al.: A knowledge-grounded neura conversation model. In: Proceedings of the AAAI Conference on Artificial Intelligence, pp. 5110–5117 (2018)
Zhu, W., Mo, K., Zhang, Y., Zhu, Z., Peng, X., Yang, Q.: Flexible end-to-end dialogue system for knowledge grounded conversation. CoRR, abs/1709.04264 (2017)
Moon, S., Shah, P., Kumar, A., Subba, R.: Opendialkg: Explainable conversational reasoning with attention-based walks over knowledge graphs. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp. 845–854 (2019)
Lin, X., Jian, W., He, J., Wang, T., Chu, W.: Generating informative conversational response using recurrent knowledge-interaction and knowledge-copy. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 41–52 (2020)
Wang, W., Gao, W., Feng, S., Chen, L., Wang, D.: Adaptive posterior knowledge selection for improving knowledge-grounded dialogue generation. In: Proceedings of the 30th ACM International Conference on Information and Knowledge Management, pp. 1989–1998 (2021)
Liu, Z., Niu, Z. Y., Wu, H., Wang, H.: Knowledge aware conversation generation with explainable reasoning over augmented graphs. In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pp. 1782–1792 (2019)
Wang, J., Liu, J., Bi, W., Liu, X., He, K., Xu, R., Yang, M.: Improving Knowledge-Aware Dialogue Generation via Knowledge Base Question Answering. In: Proceedings of the 34th AAAI Conference on Artificial Intelligence, pp. 9169–9176 (2020)
Dinan, E., Roller, S., Shuster, K., Fan, A., Auli, M., Weston, J.: Wizard of Wikipedia: Knowledge-Powered Conversational Agents. In: Proceedings of the 7th International Conference on Learning Representations, (2019)
Wang, Y., Wang, Y., Lou, X., Rong, W., Hao, Z., Wang, S.: Improving Dialogue Response Generation Via Knowledge Graph Filter. In: 2021 IEEE International Conference on Acoustics, Speech and Signal Processing(ICASSP), pp. 7423–7427 (2021)
Zhong, P., Liu, Y., Wang, H., Miao, C.: Keyword-guided neural conversational Model. In: Proceedings of the AAAI Conference on Artificial Intelligence, pp. 14568–14576 (2021)
Cho, K., et al.: Learning phrase representations using RNN encoder-decoder for statistical machine translation. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, pp. 1724–1734 (2014)
Zhao, T., Zhao, R., Eskenazi, M.: Learning discourse-level diversity for neural dialog models using conditional variational autoencoders. In: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, pp. 654–664 (2017)
Kullback, S., Leibler, R.A.: On information and sufficiency. The annals of mathematical statistics, pp. 79–86 (1951)
Luong, M.T., Pham, H., Manning, C.D.: Effective approaches to attention-based neural machine translation. CoRR, abs/1508.04025 (2015)
Gu, J., Lu, Z., Li, H., Li, V.O.: Incorporating copying mechanism in sequence-to-sequence learning. In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, pp. 199–208 (2016)
Bordes, A., Usunier, N., Garcia-Duran, A., Weston, J., Yakhnenko, O.: Translating embeddings for modeling multi-relational data. In: Advances in Neural Information Processing Systems, pp. 2787–2795 (2013)
Serban, I. V., Sordoni, A., Bengio, Y., Courville, A., Pineau, J.: Hierarchical neural network generative models for movie dialogues. CoRR, abs/1507.04808, pp. 434–441 (2015)
Liu, C. W., Lowe, R., Serban, I. V., Noseworthy, M., Charlin, L., Pineau, J.: How not to evaluate your dialogue system: an empirical study of unsupervised evaluation metrics for dialogue response generation. In: Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pp. 2122–2132 (2016)
Tian, Z., Yan, R., Mou, L., Song, Y., Feng, Y., Zhao, D.: How to make context more useful an empirical study on context-aware neural conversational models. In: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, pp. 231–236 (2017)
Li, J., Galley, M., Brockett, C., Gao, J., Dolan, B.: A diversity-promoting objective function for neural conversation models. In: Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 110–119 (2015)
Mou, L., Song, Y., Yan, R., Li, G., Zhang, L., Jin, Z.: Sequence to backward and forward sequences: a content-introducing approach to generative short-text conversation. In: Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, pp. 3349–3358 (2016)
Acknowledgements
This work was supported by the National Key R &D Program of China under Grant No. 2019YFF0302601.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Fu, D., Zhang, C., Yu, J., Sun, Q., Zhan, Z. (2022). Improving Dialogue Generation with Commonsense Knowledge Fusion and Selection. In: Memmi, G., Yang, B., Kong, L., Zhang, T., Qiu, M. (eds) Knowledge Science, Engineering and Management. KSEM 2022. Lecture Notes in Computer Science(), vol 13368. Springer, Cham. https://doi.org/10.1007/978-3-031-10983-6_8
Download citation
DOI: https://doi.org/10.1007/978-3-031-10983-6_8
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-10982-9
Online ISBN: 978-3-031-10983-6
eBook Packages: Computer ScienceComputer Science (R0)