maritime qa system and multimodal event detection (RUSH)
Key Points
- DAIRNet theme: Autonomous Processing and Reasoning
- Monash University
- Point of contact: Dr Yuan-Fang Li
bottom line up front
Problem: Maritime QA system aims at enhancing human-machine communications. For converting human natural language into a machine-understandable language (e.g., SQL queries), a semantic parser is needed. It requires a very large amount of high-quality manually-annotated data for training.
Achievement: Possibility of training a semantic parser with limited manually-annotated data enabling zero-shot training.
Benefit to Defence: In many real-world Defence scenarios, it is infeasible to obtain large amount of training data, i.e. human utterances paired with complex logical forms. Our approach addresses this issue.
Next steps: Adding other modalities like images/videos.
problem addressed
What?
- Converting human utterances into a machine-understandable language (e.g., SQL queries).
- Non-availability of a large-scale manually-annotated data for training a semantic parser, in many real-world Defence scenarios.
How?
We propose to design compact grammars to simultaneously generate both (i) logical forms, and (ii) canonical utterances, hence synthesizing the training dataset.
outcomes
- A domain-adaptive QA System.
- A web-based user interface for the backend QA system is further developed and delivered to Defence.
- An effective approach for generating large-scale domain specific training dataset for Question Answering (QA) systems.
- We are interested in endowing the situational awareness system with a capability to assist human decision makers by allowing them to interact with the system using human natural language.
big picture for Defence
Ultimate Goal
- Augment the intelligence of intelligence analysts using cutting edge AI approaches
- To provide intelligence analysts with knowledge and inferences using advanced AI approaches, helping them to improve the efficiency and effectiveness.
- This is achieved by a hybrid neuro-symbolic approach.
publications
- Shiri, F, Zhuo, TY, Li, Z, Nguyen, V, Pan, S, Wang, W, . . . Li, Y-F 2022, ‘Paraphrasing Techniques for Maritime QA system’, 2022 25th International Conference on Information Fusion (FUSION), IEEE, pp. 1-8. DOI: 10.48550/ARXIV.2203.10854
- Shiri, F, Wang, T, Pan, S, Chang, X, Li, Y-F, Haffari, R, . . . Yu, S 2021, ‘Toward the Automated Construction of Probabilistic Knowledge Graphs for the Maritime Domain’, 2021 IEEE 24th International Conference on Information Fusion (FUSION), IEEE, pp. 1-8. DOI: 10.23919/FUSION49465.2021.9626935
