coupled self-supervised learning and deep reasoning for improved processing of noisy and dynamic multimodal data from multiple sources
Key Points
- DAIRNet Theme: Patterns in Noisy and Dynamic Data
- Organisation: Deakin University
- Point of contact: A/Prof Truyen Tran
bottom line up front
The project focuses on handling multi-source, multi-modal, and irregularly timed data that covers a long duration, as might be expected for Defence applications. To tackle this problem, a new framework will be developed to:
- learn data representation, identifying low- and high-order correlations across sources and time through self-supervision without human manual annotation, and
- enhance cross-channel reasoning, deepening the understanding of the modelled system to improve dependency inference and prediction over long time scales.
The combined implementation of these components will enable Defence to make real-time informed decisions using a variety of sensing options.
Phase 1, including a comprehensive literature survey and a pilot study to model multivariate time-series for forecasting, has been completed. Over the next two years, a prototype will be developed to represent temporal data across channels, preparing the data for downstream tasks such as anomaly detection, forecasting, and event prediction. A further component will enable complex reasoning across channels and through time. The system will be tested using two Defence-relevant case studies on anomaly detection on a naval platform and building cyber situational awareness through analysing network traffic exchanged by assets.
problem addressed
We will develop a new neural network system that exhibits the dual “fast and slow thinking” capability. The “fast thinking” mode rapidly transforms raw dynamic data into an abstract representation suitable for simple tasks, while the “slow thinking” mode performs deliberate reasoning for complex tasks across sources and time.
outcomes
We have created a prototype for forecasting multi-channel data streams and tested it on traffic flow, electricity demand, and solar power production. In the coming months, the capabilities of this prototype will be expanded to handle complex scenarios, including high-order correlations between data channels, multiple sampling rates, and a mix of continuous signals and discrete static data. Then, a subsystem will be added to allow for more intricate reasoning over a longer duration across sources.
the big picture for Defence
We will demonstrate the models developed on relevant Defence problems with DST domain experts. We will conduct two case studies, each lasting about 3 months, in collaboration between DST and A2I2 members. A2I2 will propose the case studies for consideration by DAIRNet representatives and provide training if needed. The models and prototypes will be eventually transferred to Defence to address cross-domain Defence problems with multiple disciplines. In addition to the technology development in Phase 2 and the two Defence-specific case studies, A2I2 personnel will be available for additional in-depth application studies upon request.
