Key Idea
What if advisor could see your screen?
Evaluation of Distraction Detection
How accurately can INA detect off-task behavior?
To find out, we built IntentionBench , a benchmark dataset that combines focused sessions into realistic mixed sessions containing both on-task and off-task activity. We then evaluate how well INA detects transitions between the two.
INA achieved high accuracy and F1 score.
Based on these results, we moved on to a real-world user study.
Clarification and feedback jointly improve distraction detection.
| Clarification | Feedback | Accuracy | Precision | Recall | F1 |
|---|---|---|---|---|---|
| – | – | 0.805 | 0.897 | 0.629 | 0.739 |
| ✓ | – | 0.871 | 0.949 | 0.748 | 0.836 |
| – | ✓ | 0.845 | 0.959 | 0.677 | 0.794 |
| ✓ | ✓ | 0.878 | 0.959 | 0.755 | 0.845 |
The check mark (✓) indicates whether clarification or feedback is included; the best performance is in bold. Employing both features shows the best balance. The bottom row is our deployed configuration.
We conducted a three-week user study with 22 users, comparing three systems during everyday computer use.
* To minimize bias, all systems were presented to participants using color-based names: Orange, Blue, and Purple.
Experiment
Orientation
Post-survey
Post-interview
Post-survey
Post-interview
Post-survey
Post-interview
Participants completed the same post-survey after each weekly condition. Closed-ended items used a 1–5 Likert scale.
The program understood my intentions.
The program helped me act on the activities I intended to do.
I felt supported by the program.
The program made me feel capable of controlling my digital activities.
The program used my data in a meaningful way.
I worried my personal information might leak.
The program gave me new insights into my computer use.
Using this program interfered with my workflow. Reverse-scored
Notifications helped me act according to my intentions.
Notifications motivated intended activities.
Notifications arrived at appropriate times.
The program tailored its messages to my intentions and activities.
The program provided clear explanations for its message.
Notifications felt burdensome. Reverse-scored
The program adapted its behaviour based on my feedback.
While using the computer I am able to block out most other distractions.
While using the computer, I am absorbed in what I am doing.
While on the computer, I am immersed in the task I am performing.
When on the computer, I get distracted by other attentions very easily. Reverse-scored
While on the computer, my attention does not get diverted very easily.
Was there a moment when the program helped you focus or carry out tasks as intended? If so, please explain.
Has the program helped you focus or carry out tasks as intended? If so, please describe with examples, including the context and how the program helped.
What did you like or dislike about the program?
What insights, if any, have you gained about your digital habits or computer use?
Three-week within-subjects study with 22 participants. INA reduced LLM-estimated off-task ratio from 0.166 to 0.104 compared with simple reminder (p < .001), and increased intention alignment from 4.23 to 4.44 (p < .001).
Focused immersion was 3.74 for INA, 3.34 for simple reminder, and 2.90 for logging only. The study logged 1,786 sessions, 1,360 hours of usage, and 2,449,690 context records.
The weekly survey reports category-level scores. INA scored 3.75 on Support, compared with 3.56 for simple reminder and 2.75 for logging only. Message Effectiveness was 3.74 for INA and 3.24 for simple reminder.
Workflow Disruption was lower for INA (3.09) than both baselines (3.91), indicating that the more active assistant introduced interruptions that future designs should tune.
INA can evolve into an ambient assistant that supports intentional living implicitly across devices, including smart glasses, without requiring explicit intent input.
INA may reinforce , as the LLM predicts intent rather than executing it, bypassing standard refusal mechanisms. Integrating external models can mitigate this risk.
@inproceedings{choi2026state,
title = {State Your Intention to Steer Your Attention: An AI Assistant for Intentional Digital Living},
author = {Choi, Juheon and Lee, Juyong and Kim, Jian and Kim, ChanYoung and Min, Taywon and Knox, W. Bradley and Lee, Min Kyung and Lee, Kimin},
booktitle = {Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems},
pages = {1--29},
year = {2026}
}