Novel Perception

Abstract

The Novel Perception system enhances users’ awareness of the world by providing them naturalistic access to sources of information beyond what is perceivable by the basic human senses. The system, conceived of and designed by a team at the Johns Hopkins University Applied Physics Laboratory (APL), collects signals from a variety of sensors, synchronizes them in real time, registers them in real space, and then overlays them onto the real world as imagery and holograms seen through a mixed reality (MR) headset. The concept includes “virtual lenses” of hyperspectral, radio frequency, social, physiological, thermal, and radiological/nuclear overlays, so that a user can select multiple virtual lenses to create on-the-fly custom compound lenses across these modalities. Because the volume and velocity of the data streaming from these sensor modalities may be overwhelming, the system is envisioned to leverage artificial intelligence and brain–computer interfaces to sculpt the deluge per the operational tasks at hand. This article describes the approach to developing the system as well as possible applications.