Bill Qian

When training AI agents to perform tasks, humans often care not only about whether a task is completed but also how it is performed. As agents tackle increasingly complex tasks, aligning their behavior with human-provided specifications becomes critical for responsible AI deployment. Reward design provides a direct channel for such alignment by translating human expectations into reward functions that guide reinforcement learning (RL). However, existing methods are often too limited to capture nuanced human preferences that arise in long-horizon tasks. Hence, we introduce Hierarchical Reward Design for Language (HRDL): a problem formulation that extends classical reward design to encode richer behavioral specifications for Hierarchical RL agents. We further propose Language to Hierarchical Rewards (L2HR), our proposed solution to HRDL. Human subject and numerical experiments show that Hierarchical RL agents trained with rewards designed via L2HR not only complete tasks effectively but also better adhere to human specifications. Together, HRD and L2HR advance the research on human-aligned AI agents.

Human collaborator’s workload plays a central role in human-robot collaboration. Algorithms designed to minimize cognitive workload enhance fluent human-robot teamwork. Time series data of workload is vital for both designing and assessing these algorithms. However, accurately quantifying and measuring cognitive workload, particularly at high temporal resolution, poses a substantial challenge. Towards addressing this challenge, we explore the potential of after-action reviews (AARs) as a tool for gauging workload during human-robot collaboration. First, through a case study, we present and demonstrate AutoAAR for measuring human workload post-task at a high temporal resolution. Second, through a user study, we quantify the validity and utility of measurements derived using AutoAAR for human-robot teamwork. The paper concludes with guidelines and future directions to extend this method to measure other internal states, such as trust and intent.

To forge effective collaborations with humans, robots require the capacity to understand and predict the behaviors of their human counterparts. There is a growing body of computational research on human modeling for human-robot interaction (HRI). However, a key bottleneck in conducting this research is the relative lack of data of cognitive states – like intent, workload, and trust – which undeniably affect human behavior. Despite their significance, these states are elusive to measure, making the assembly of datasets a challenge and hindering the progression of human modeling techniques. To help address this, we first introduce Rescue World for Teams (RW4T): a configurable testbed to simulate disaster response scenarios requiring human-robot collaboration. Next, using RW4T, we curate a multimodal dataset of human-robot behavior and cognitive states in dyadic human-robot collaboration. This RW4T dataset includes state, action and reward sequences, and all the necessary data to replay a visual task execution. It further contains psychophysiological metrics like heart rate and pupillometry, complemented by self-reported cognitive state measures. With data from 20 participants, each undertaking five human-robot collaborative tasks, this dataset (comprising of 100 unique trajectories) accompanied with the simulator can serve as a valuable benchmark for human behavior modeling.

The COVID-19 pandemic has emphasized the importance of accurate detection of known and emerging pathogens. However, robust characterization of pathogenic sequences remains an open challenge. To address this need we developed SeqScreen, which accurately characterizes short nucleotide sequences using taxonomic and functional labels and a customized set of curated Functions of Sequences of Concern (FunSoCs) specific to microbial pathogenesis. We show our ensemble machine learning model can label protein-coding sequences with FunSoCs with high recall and precision. SeqScreen is a step towards a novel paradigm of functionally informed synthetic DNA screening and pathogen characterization, available for download at www.gitlab.com/treangenlab/seqscreen.