TECHNOLOGY AREA(S): Information Systems

ACQUISITION PROGRAM: IWS 5.0, Undersea Warfare Systems

OBJECTIVE: Develop a suite of video processing algorithms utilizing the machine learning (ML) techniques of artificial intelligence (AI) reinforcement learning, deep learning, and transfer learning to process submarine imagery obtained by means of periscope cameras.

DESCRIPTION: The Navy, across all platforms, generates huge amounts of data to process and requires efficient, high-performing tools to extract information that will reduce the amount of effort needed by human operators to assess the data. Periscope imagery is one class of data where human failure to adequately assess the data available can be catastrophic. ML is one approach that will address this challenge.

AI approaches such as ML [Ref 1] often utilize reinforcement learning, deep learning, and transfer learning, but generally have not utilized all approaches in an overall system. ML tools are being deployed across many commercial and Department of Defense (DoD) products, but these tools are usually deployed as ‘black boxes,’ with limited understanding of how the approaches work. In these cases, performance is only characterized as a function of available training data. However, in the case of DoD data, such as Navy periscope imagery, the data available to train a black box is not robustly representative of the range of imagery expected to be encountered across all operating conditions. Pre-tuned black box approaches are therefore not suited to the Navy imagery challenge.

Reinforcement and transfer learning algorithms are desired to address video processing within DoD communities in cases where available training data is not sufficient to support black box approaches which may utilize deep learning as the initial approach.

The Navy seeks innovation in the simultaneous use of reinforcement [Refs 2, 3] and transfer learning [Ref 4] as a means of developing effective algorithms for processing complex video data that varies significantly over time and environments, as occurs in the case of submarine periscope imagery. Despite collecting large amounts of video data with 360-degree cameras operating at frame rates of 60 fps or higher, available recorded data represents a sparse sampling of the range of conditions and vessel traffic that submarine periscopes could be expected to encounter across the Fleet. Effective analysis of periscope data requires algorithms that evolve over time to adapt to new environments. The Navy also seeks innovation regarding how transfer learning can effectively share complex imagery data and algorithms between boats and shore sites in the face of limited communication opportunities and bandwidth.

The envisioned outcome of this effort is a suite of ML algorithms that can work with a relatively sparse training set. This suite of algorithms should address particular periscope processing problems, such as timely vessel detection, identification, and re-acquisition. Key metrics involve latency of vessel detection, time to identify, latency of vessel re-acquisition after loss, rate of false positives, and rate of missed identifications. The suite of ML algorithms would then need to utilize reinforcement learning to improve system performance over time, following initial certification and fielding via standard military capability fielding paradigms. The improvements acquired over time would then be shared with other submarine platforms using transfer-learning algorithms to propagate evolutionary system improvements across the Fleet. Additionally, the algorithms must be capable of real time processing (30 to 60 frames per second) utilizing one or two graphical processing units. Testing of systems will be performed using previously collected imagery in a software development environment.

Improvements developed under this STTR topic will be incorporated into fielded imagery systems starting with improvements to the submarine periscope imagery system, which is updated every two years through the IWS 5.0 Advanced Cross-platform Build (AxB) development process. Ability to improve capability through software will eliminate hardware-related lifecycle costs, with potential to reduce total lifecycle costs due to improved performance coupled with shared learning.

PHASE I: Develop a concept for a suite of video processing algorithms. Demonstrate the concept can feasibly meet the requirements of the Description to use reinforcement and transfer learning to improve system performance and update the system with results. Establish feasibility through modeling and analysis of the algorithms using representative imagery data (which will be provided). The Phase I Option, if exercised, will include the initial design specifications and capabilities description to build a prototype solution in Phase II.

PHASE II: Develop a prototype of the suite of video processing algorithms and deliver for independent laboratory evaluation by the Navy. Validate the prototype through testing to demonstrate improvements relative to individual performance metrics as well as computation of mission performance metrics as defined in the Description. Provide a detailed test plan to demonstrate the prototype achieves the metrics defined. Develop a Phase III plan.

PHASE III DUAL USE APPLICATIONS: Support the Navy in transitioning the technology to Navy use by working with the IWS 5.0 AxB development process to further assess system performance and integrate Phase II results into relevant platform hardware. The AxB development process will utilize many of the same metrics utilized during the STTR effort, but will add an effort to integrate the products into the appropriate submarine system, with the algorithm developer working with a prime integrator. The new tools will also be assessed in terms of operator impact, if it decreases overall workload.

Vehicle cameras are being used to avoid collisions and are being used to support self-driving cars. Digital cameras and cell phones now detect faces reliably. Networked cloud applications like Facebook and Google Images can identify scenes and individuals in photos. While commercial applications rarely suffer from the limited communication and bandwidth associated with submarines, development of new tools that leverage both reinforcement learning and transfer learning should be extensible to a variety of potential applications to provide improvements in these other video processing applications.

REFERENCES:

1. Soria Olivas, Emilio. et al. (editors.), “Handbook of Research on Machine Learning Applications and Trends: algorithms, methods, and techniques.” Information Science Reference, Hershey, PA, 2010. https://www.worldcat.org/title/handbook-of-research-on-machine-learning-applications-and-trends-algorithms-methods-and-techniques/oclc/315237547

2. Sutton, Richard, and Barto, Andrew. “Reinforcement learning: an Introduction”. MIT Press, Cambridge, MA, 2018. https://www.worldcat.org/title/reinforcement-learning-an-introduction/oclc/1030967380

3. Heess, N. et al. “Learning continuous control policies by stochastic value gradients.”  NIPS Proceedings, 2015: pp. 2944-2952. https://www.worldcat.org/title/learning-continuous-control-policies-by-stochastic-value-gradients/oclc/6052614242

4. Carroll, J. L. and K. Seppi. “Task similarity measures for transfer in reinforcement learning task libraries.” Proceedings of the 2005 IEEE International Joint Conference on Neural Networks, Vol.2, 2005; pp. 803-808.  https://www.worldcat.org/title/proceedings-of-the-international-joint-conference-on-neural-networks-ijcnn-2005-july-31-august-4-2005-hilton-montreal-bonaventure-hotel-montreal-quebec-canada/oclc/62778290

KEYWORDS: Machine Learning; Transfer Learning; Reinforcement Learning; Deep Learning; Artificial Intelligence; Video Processing of Periscope Imagery; ML; AI