Handows: A Palm-Based Interactive Multi-Window Management System in Virtual Reality Jin-Du Wang , Ke Zhou , Haoyu Ren , Per Ola Kristensson , and Xiang Li Window Closure X Window Scaling Window Selection Windows in VR Window Positioning Handows Fig. 1: In this paper, we present Handows—a palm-based interactive window management system designed for virtual reality (VR). It enables users to control multiple spatial windows in VR environments using their non-dominant palm as an interactive surface. The system supports four core operations: (1) window selection, (2) window closure, (3) window positioning, and (4) window scaling. Abstract—Window management in virtual reality (VR) remains a challenging task due to the spatial complexity and physical demands of current interaction methods. We introduce Handows, a palm-based interface that enables direct manipulation of spatial windows through familiar smartphone-inspired gestures on the user’s non-dominant hand. Combining ergonomic layout design with body-centric input and passive haptics, Handows supports four core operations: window selection, closure, positioning, and scaling. We evaluate Handows in a user study (N = 15) against two common VR techniques (virtual hand and controller) across four core window operations. Results show that Handows significantly reduces physical effort and head movement while improving task efficiency and interaction precision. A follow-up case study (N = 8) demonstrates Handows’ usability in realistic multitasking scenarios, highlighting user-adapted workflows and spontaneous layout strategies. Our findings also suggest the potential of embedding mobile-inspired metaphors into proprioceptive body-centric interfaces to support low-effort and spatially coherent interaction in VR. Index Terms—Window management, layout, virtual reality, on-body interaction, direct manipulation 1 INTRODUCTION One of the major promises of virtual reality (VR) environments is their ability to overcome physical constraints and offer virtually unlimited display space [52]. Unlike traditional monitors or mobile devices, VR systems allow users to open, position, and manipulate multiple applica- tion windows of varying sizes throughout a 3D virtual environment [23]. This spatial flexibility is especially appealing for multitasking and information-rich workflows. However, the current state of window management in VR is still limited in both usability and efficiency [36]. • Jin-Du Wang is with The Hong Kong University of Science and Technology. E-mail: jwangki@connect.ust.hk. • Ke Zhou is with Chongqing University. E-mail: kezhou@stu.cqu.edu.cn. • Haoyu Ren is with Xi’an Jiaotong University. E-mail: 2206113950@stu.xjtu.edu.cn. • Per Ola Kristensson and Xiang Li are with the University of Cambridge. E-mail: E-mail: {pok21, xl529}@cam.ac.uk. • Xiang Li is the corresponding author. Manuscript received xx xxx. 201x; accepted xx xxx. 201x. Date of Publication xx xxx. 201x; date of current version xx xxx. 201x. For information on obtaining reprints of this article, please send e-mail to: reprints@ieee.org. Digital Object Identifier: xx.xxxx/TVCG.201x.xxxxxxx In traditional computing environments, such as desktops, laptops, and smartphones, window management is supported by mature 2D graphical user interfaces (GUIs), which rely on the well-established WIMP model: windows, icons, menus, and pointers [9, 17]. These sys- tems enable a wide range of functionalities, including window creation, movement, resizing, and layout customization [57]. Multi-window management is further supported by features like task overviews, lay- out preservation, and rapid switching between windows [47, 56, 58]. Critically, the compatibility between these GUI elements and 2D input methods, such as mice or touchscreens, enables efficient interaction with minimal physical or cognitive effort [1]. In contrast, VR platforms generally provide only basic interac- tion techniques for spatial windows [6, 7], often relying on ray- casting with controllers or mid-air gestures for selection and manipula- tion [24, 33, 54]. Although these methods offer spatial freedom, they often require large arm movements, precise coordination, and repetitive actions, which can lead to fatigue [23]. Moreover, the absence of tactile feedback and limited layout support hinders fluid multitasking and distracts users from their primary goals [54]. To address these challenges, we introduce Handows, a novel win- dow management system designed specifically for VR environments. Handows is a palm-based interface situated on the user’s non-dominant hand, allowing direct interaction with window thumbnails through fa- https://orcid.org/0009-0009-4028-4662 https://orcid.org/0009-0007-9931-1616 https://orcid.org/0009-0004-5954-1167 https://orcid.org/0000-0002-7139-871X https://orcid.org/0000-0001-5529-071X miliar gestures such as tapping, swiping, and pinch-to-zoom. Drawing inspiration from mobile interaction paradigms and grounded in princi- ples of human-computer interaction (HCI) [5,45,46], Handows aims to reduce physical strain and cognitive effort by integrating a miniature interface into the user’s proprioceptive space. It also introduces a de- fault ergonomic layout for organizing windows based on angular and distance-related comfort zones [28, 36]. To evaluate the effectiveness of Handows, we conducted a two- part investigation. First, a user study with 15 participants compared Handows with a virtual hand interface and a controller-based baseline across four fundamental window operations: selection, closure, posi- tioning, and scaling. Performance metrics included task completion time, accuracy, head and hand movement, and subjective preference. Second, we carried out a case study simulating a realistic multitask- ing scenario (i.e., trip planning and budgeting) to examine how users employ Handows in continuous, goal-driven workflows. Findings from both studies reveal that Handows significantly im- proves task efficiency, reduces physical effort, and enhances user satis- faction. In the user study, Handows outperformed the other techniques in selection and closure tasks, reduced head rotation by over 60%, and achieved the most precise scaling performance. In the case study, participants adapted familiar mobile-inspired interaction patterns and spontaneous layout strategies, reporting that the palm-based interface aligned closely with their interaction habits and supported fluid multi- tasking in immersive environments. In summary, this paper makes the following contributions: • We introduce Handows, a palm-based VR window management system that integrates spatial miniaturization, passive haptics, and mobile-inspired gesture interaction within the user’s propriocep- tive space. • We demonstrate that embedding mobile-inspired metaphors into proprioceptive, body-centric interfaces improves user perfor- mance and reduces perceived effort in window management tasks in VR. • We identify how users adapt spatial layouts and develop sustained interaction strategies in a realistic multitasking scenario, reveal- ing how body-anchored interfaces can support fluid, goal-driven workflows in immersive environments. 2 RELATED WORK We situate Handows within three strands of prior research: (1) window management techniques in virtual environments, (2) spatial represen- tation and miniature views, and (3) on-body interaction interfaces. Together, these threads inform the ergonomic, spatial, and interaction design of our system. 2.1 Window Management in Virtual Environments In VR systems, application content is typically rendered as floating 2D windows within a 3D space [23], which introduces new challenges for interaction and spatial organization [14, 36]. Various input tech- niques have been explored. Direct hand-based interaction has been widely explored, but physical reach limitations and fatigue constrain its scalability [14]. More commonly, ray-casting is used—either via hand-held controllers or gesture recognition [24, 33, 54]—though these methods lack fine control and tactile grounding. To improve manipu- lation, Projective Windows [36] introduced continuous hand gestures for window interaction, while cross-device AR systems have enabled basic window control via smartphones [55]. However, many of these approaches remain limited by their reliance on frequent arm movement and the absence of haptic cues. Spatial layout also significantly impacts user performance and com- fort. Egocentric layouts can reduce motion sickness and improve task efficiency [20], and users generally prefer multi-window configurations for complex tasks [4]. While some systems aggregate windows into large immersive displays [51], others recommend curved or segmented arrangements to support peripheral awareness and multitasking [60]. Recent work has also explored leveraging underutilized peripheral space and gaze-based cursor teleportation to improve window switch- ing efficiency on large virtual displays [50], though such techniques still operate within conventional desktop interaction paradigms and offer limited ergonomic rethinking. Dynamic and context-aware win- dow placements have also been proposed [16, 25], but such designs can increase the difficulty of spatial recall and require users to adapt to inconsistent spatial mappings [16]. 2.2 Miniature Representations and Spatial Navigation To support navigation and control in virtual spaces, several systems have introduced miniature representations. The World-in-Miniature (WIM) technique [61] enables users to manipulate the environment at a reduced scale. Similarly, multi-viewport systems and minimap-style interfaces assist in orienting users to off-screen targets in both 2D and 3D [12, 19, 27, 65]. These representations improve global awareness but are often limited by visibility constraints or require additional workspace that may conflict with active content. While valuable for scene navigation, they are less frequently used for fine-grained content manipulation such as window control. 2.3 On-Body Interaction and Passive Haptics On-body interaction offers a promising alternative to traditional mid- air gestures by leveraging proprioception and passive haptic feedback. Prior work has demonstrated that skin surfaces can serve as effective input regions, improving accuracy and reducing fatigue [22, 37, 39, 48, 49]. Systems like SkinWidget [2] and others [30, 31] implement forearm- and wrist-based input, although fatigue remains a concern with extended use. Similarly, microGEXT [38] leverages the side of the finger for sliding gestures, enabling self-haptic feedback during text selection and editing in VR. Palm-based interaction, in particular, has shown strong potential due to its visual accessibility and biomechanical stability. Several projects have proposed projecting interactive elements onto the palm [15,62,64], while others have explored imaginary touch interfaces [29] or input techniques such as PalmGesture [63] and on-body menus [3,37]. These works suggest that the palm can act as a natural, proprioceptively- aligned interaction surface. STAR [35] demonstrates how smartphone- like interactions can be effectively mapped to the hand, pointing to opportunities for integrating familiar 2D paradigms into immersive contexts. In summary, our work builds on insights from VR window man- agement, spatial navigation tools, and on-body input research. While prior systems have explored the palm as an interactive surface and used miniature representations for spatial navigation, the novelty of Handows lies in its specific synthesis of these concepts for the com- plex task of multi-window management in VR. Handows contributes a unique combination of spatial miniaturization, passive haptics from the user’s own body, and familiar smartphone-inspired gestures, all situated within an ergonomically optimized layout. This integrated, body-centric approach is designed to provide more usable and efficient VR window control. 3 HANDOWS: A WINDOW MANAGEMENT SYSTEM FOR VR Modern operating systems provide robust and intuitive solutions for managing multiple application windows. On Windows PCs, Task View enables users to overview and switch between tasks using a structured, tiled layout. macOS offers Mission Control, which presents all open windows spatially across desktops, facilitating spatial reasoning and task switching. On mobile platforms, both Android and iOS implement App Switcher views that support fluid navigation across applications through touch gestures like swipe and tap (see Figure 2). These existing paradigms emphasize spatial structure and rapid ac- cess that we aim to bring into immersive environments. Inspired by these systems, we introduce Handows, a palm-based window man- agement system designed specifically for VR. Handows addresses the spatial, ergonomic, and attentional challenges of managing multiple windows in 3D by anchoring the interface on the user’s non-dominant palm. The system integrates spatial miniaturization, passive haptic feed- back, and gesture-based control to support fast, embodied interaction Task View App Switcher Mission Control Fig. 2: Common multi-window management paradigms across platforms. Task View (Windows) and Mission Control (macOS) provide spatial overviews of open windows using mouse input. On mobile platforms, the App Switcher allows users to navigate recent tasks through swipe gestures. These systems inspired the design of Handows, which adapts structured layout and gesture-based control into a palm-based interface for VR. with minimal fatigue. At its core, Handows consists of two key compo- nents: (1) a default spatial layout that helps users organize and position multiple windows efficiently in the environment, and (2) a palm-based miniature interface that supports core operations—window selection, closure, positioning, and scaling—through smartphone-inspired ges- tures. 3.1 Spatial Layout Optimization While VR enables the placement of an unlimited number of applica- tion windows throughout 3D space, unrestricted spatial freedom often leads to disorganized layouts, increasing cognitive load and interac- tion complexity [42]. To mitigate these issues, Handows adopts a default window layout designed to promote visual clarity, reduce head movement, and enhance access efficiency. The system follows a multi-display strategy in which each applica- tion occupies an independent virtual window, rather than combining content into a unified display surface [4, 26]. Drawing from ergonomic guidelines on monitor positioning [13, 44], Handows places windows within the user’s natural field of view, constrained by angular and distance thresholds to reduce visual fatigue. The central window is re- served for the user’s primary task, while secondary windows are placed at similar or higher elevations, avoiding vertical overlap with physical furniture such as desks [52]. This spatial configuration maintains con- sistency with real-world expectations while minimizing interference and supporting fluid transitions between windows. 3.1.1 Curved Layout Selection To refine the spatial configuration, we conducted an in-lab design pa- rameter study (N = 8) comparing four alternative layout geometries: flat, horizontally curved, vertically curved, and both horizontally and vertically curved. Eight participants in a controlled lab setting evalu- ated each layout in terms of usability and peripheral search efficiency. Following Barrett et al.’s task design [21], participants performed target- counting tasks involving color-shape combinations across peripheral displays. Each participant completed 20 trials under each layout con- dition. Layout order was counterbalanced to mitigate ordering effects. Our results indicated a consensus preference for the combined hori- zontal and vertical curvature layout. Participants reported improved visibility and reduced neck rotation during target acquisition. Based on these findings, Handows adopts this curved configuration as the system’s default spatial arrangement (see Figure 3). 3.2 Palm-Based Interface and Interaction Design Handows situates its interface on the user’s non-dominant palm, em- ulating the size and feel of a smartphone screen. This body-centric approach offers both proprioceptive alignment and passive haptic feed- back, enhancing spatial awareness and reducing visual dependence during interaction. The interface is activated when the palm faces up- ward and displays a grid of window thumbnails, each representing a currently open application. This layout supports simultaneous access to multiple windows without overwhelming the user’s visual field. Fig. 3: Representations of the default layout (a) Side view, (b) Top view. All screens are placed within an area where the user’s head rotation does not exceed 45 degrees to the left or right and 10 degrees upward [44]. The primary screen is positioned in the optimal area [13,52], and each screen is 19 inches (a common size of displays). Fig. 4: Left: Three surfaces providing passive haptic feedback with operational panels (black areas) for virtual reality interactions: (a) Single Palm Surface (opted in Handows), (b) Nested Finger Surface, (c) Index Finger Surface. Right: Overview of the Handows interface, illustrating the comprehensive layout and functional components. We considered alternative placements, including finger-based sur- faces and nested configurations [35] (see Figure 4 (b) and (c)), but selected the palm interface following exploratory testing, which in- dicated advantages in interaction area, posture stability, and visual accessibility. The interface encourages a natural phone-holding posture, facilitating prolonged use while minimizing fatigue. 3.3 Features Handows supports four core window operations: (1) selection; (2) closure; (3) positioning; and (4) scaling (see Figure 1), by adapting familiar mobile gestures to embodied interaction in VR. All interactions are performed using the dominant hand, while the non-dominant palm serves as a stable, proprioceptively accessible control surface. This bimanual configuration minimizes fatigue and supports fluid transitions between operations. • Selection: Users tap a window thumbnail on the palm interface to bring the corresponding window into focus. • Closure: A swiping gesture toward the edge of the palm removes the selected window from the layout. • Positioning: Dragging a thumbnail repositions the associated window, which automatically snaps to the nearest predefined slot in the ideal spatial layout. • Scaling: Pinch-to-zoom gestures are used to resize windows, mimicking zoom interactions on smartphones. By integrating well-established gesture paradigms within a body- centric interface, Handows enables intuitive and efficient window man- agement in immersive environments. This combination of mobile famil- iarity, passive haptic feedback, and ergonomic layout design supports low-effort, high-control interaction across a range of VR workflows. To ensure a rigorous evaluation, the system’s scope is deliberately fo- cused on the manipulation of existing windows. This choice avoids the complex and confounding variables tied to new window creation. Supporting the latter would require addressing broader issues, such as where new windows should appear, how to resolve overlaps, and how to group or order windows, which we consider beyond the scope of this work. 4 USER STUDY To assess the performance and user experience of Handows for window management tasks in VR environments, we carried out a user study comparing it with two baseline techniques prevalent in current VR systems: Virtual Hand and Controller. This investigation aimed to examine how the design characteristics of Handows, such as body- centric anchoring, spatial miniaturization, and gesture familiarity, affect user interaction compared to established methods. Our goal was to evaluate not only interaction efficiency, but also physical demand and subjective experience. Although the Handows design is applicable to both VR and MR environments, we conducted our experiments in VR to ensure environmental control and consistency, thereby strengthening the internal validity of our findings. To guide this study, we proposed the following hypotheses: H1: We hypothesized that by adapting familiar mobile gestures onto a miniaturized, palm-based interface, HANDOWS would enable faster and more precise window manipulation compared to existing techniques. H2: We expected that anchoring the interface to the user’s body would significantly reduce physical efforts and fatigue during window management tasks. H3: We anticipated that the combination of passive haptic feedback, proprioceptive stability, and the high transferability of familiar gestures would make HANDOWS a more intuitive and satisfying technique, resulting in more positive subjective experiences and stronger user preference for HANDOWS over the baselines. 4.1 Participants and Apparatus A total of 15 participants (10 males and 5 females) were recruited for the study. The age range of the participants was between 20 and 23 years (M = 20.67,SD = .72). All participants were students at a local university. All participants reported previous experience with VR, with familiarity ratings ranging from 1 to 7 on a 7-point Likert scale, where 1 indicated no experience in VR, and 7 indicated expertise (M = 3.13,SD = 1.96). All participants were right-handed habitual users. This study was approved by the XJTU research ethics board. The user study was conducted in a university laboratory equipped with a desktop computer, display devices, and an area for partici- pants to engage in VR interactions. The application, developed in Unity 2022.3.8f1, was run from the Unity Editor on a desktop PC and streamed to a Meta Quest 2 headset via a wired Quest Link connec- tion. The system utilized the Meta Interaction SDK for hand tracking. Window management operations recognition was based on detecting a collision between the fingertip of the dominant hand and a virtual panel on the non-dominant hand. This method is robust to variations in resting hand pose (e.g., whether the hand is clenched or extended) and does not rely on a specific finger configuration. Furthermore, the Meta Interaction SDK automatically estimates the user’s hand size during initialization and scales the virtual hand model accordingly. 4.2 Method and Procedure The study followed a within-subject design with one independent vari- able, WINDOW MANAGEMENT TECHNIQUE. Users were tasked with completing four objectives: target window selecting, closing, position- ing, and scaling, using three different window management techniques: Handows, Virtual Hand, and Controller. After a brief introduction, participants were given 10 minutes to familiarize themselves with the window management techniques and tasks that would be undertaken. Additionally, based on each user’s dominant hand and operating habits, it was confirmed which hand Fig. 5: The procedure of the user study. would serve as the operating hand, while the other hand would be used for placing the control panel. To minimize the influence of learning effects, the order of window management techniques assigned to participants was counterbalanced using a Latin square design. After participants completed the desig- nated tasks for each technique, they were given a two-minute break before proceeding to the next technique. After completing all tasks, participants were asked to fill out questionnaires evaluating their ex- perience with the different manipulation techniques. Further, a final comparison was included, where participants were asked to rate their preference for each manipulation technique. Finally, a semi-structured interview was conducted to gather qualitative insights, allowing partici- pants to share their strategies and provide suggestions. The process is shown in Figure 5. On average, participants spent approximately 50 minutes completing the study, including the tasks and questionnaire. Participants received a small monetary reward for their participation. 4.3 Tasks and Measurement The evaluation metrics include: (a) Task Completion Time (s), measur- ing the duration from the target window’s appearance to task comple- tion; (b) Hand Movement Distance (m), representing the total distance traveled by the participant’s dominant hand; (c) Head Rotation Angle (°), indicating the degree of head movement; and specifically for Task 4, (d) Scaling Deviation (%), which captures the difference between the actual and target window sizes to assess scaling accuracy, as suggested by Li et al. [40, 41]. We also collected subjective feedback through several questionnaires: (a) Perceived Workload, measured by the raw NASA-TLX [32]; (b) Usability, measured by the System Usability Scale (SUS) [11]; (c) User Experience, measured by the UEQ-Short [59]; (d) Perceived Fatigue, measured by the Borg 6–20 scale [8]; and (e) Simulator Sickness, mea- sured by the Simulator Sickness Questionnaire (SSQ) [34]. Finally, participants were asked to provide a comparative evaluation of the three techniques through a final questionnaire. This included: (1) rank- ing their preferred window management technique among Handows, Virtual Hand, and Controller; (2) rating the impact of passive haptic feedback from touching the hand during Handows operation on oper- ational stability (on a 7-point scale); and (3) assessing the extent to which prior experience with traditional computing devices, such as computers and smartphones, could be transferred to each technique (on a 7-point scale). 4.3.1 Task 1: Target Window Selection In this task, participants were asked to select a target window, which was highlighted in red at the start of each trial. Upon selection, the window reverts to its original color, and a new target is highlighted. Participants were instructed to perform ten selection operations for each of the six windows, totaling 180 selections per participant across three conditions. The appearance order of the target windows was randomized. 4.3.2 Task 2: Target Window Closure In this task, participants were asked to close the target window, which was highlighted in red. Upon closure, a new target window would appear among the remaining windows and would be highlighted in red. When all windows in the scene are closed, the scene resets and a Table 1: Mean values (M), standard deviations (SD), and test statistics for completion time, hand movement distance, and head rotation across the four tasks (Target Selection, Closure, Positioning, and Scaling) using three interaction techniques: HANDOWS, CONTROLLER, and VIRTUAL HAND. For each metric, either repeated-measures ANOVA (F) or Friedman test (χ2) was conducted based on the normality of data distribution. Significant p-values are reported for overall effects; post-hoc test results are detailed in the text. Task Technique Time Hand Movement Distance Head Rotation Angle M (SD) Statistics M (SD) Statistics M (SD) Statistics Task 1 Handows 0.88 (0.16) p < 0.0001 0.22 (0.07) p < 0.0001 4.67 (1.07) p < 0.0001 Controller 0.92 (0.15) F(2,28) = 28.271 0.21 (0.04) χ2(2) = 19.6 23.46 (8.11) F(2,28) = 58.831 Virtual Hand 1.18 (0.23) 0.34 (0.09) 29.78 (10.35) Task 2 Handows 1.16 (0.24) p < 0.0001 0.28 (0.10) p < 0.0001 6.42 (2.98) p < 0.0001 Controller 1.50 (0.34) F(2,28) = 96.413 0.24 (0.07) F(2,28) = 142.498 28.38 (7.78) χ2(2) = 26.53 Virtual Hand 2.53 (0.58) 0.36 (0.07) 37.20 (6.82) Task 3 Handows 1.88 (0.44) p < 0.0001 0.30 (0.12) p < 0.0001 7.38 (3.09) p < 0.0001 Controller 1.30 (0.21) F(2,28) = 28.695 0.43 (0.12) χ2(2) = 24.13 34.49 (18.55) F(2,28) = 54.255 Virtual Hand 1.97 (0.27) 0.66 (0.15) 46.24 (12.14) Task 4 Handows 2.10 (0.57) p < 0.0001 0.20 (0.08) p < 0.0001 7.76 (3.70) p < 0.0001 Controller 2.83 (0.57) F(2,28) = 110.622 0.31 (0.05) F(2,28) = 56.522 40.87 (7.50) χ2(2) = 28.13 Virtual Hand 5.06 (1.13) 0.47 (0.09) 49.41 (8.38) new target window appears among the newly generated six windows. In each condition, participants were instructed to perform 10 closing operations on each of the six windows in the space, totaling 180 closing operations per participant. The appearance order of the target windows was randomized. 4.3.3 Task 3: Target Window Positioning In this task, participants were asked to position a target window high- lighted in red to a specified position highlighted in yellow. Upon successful positioning, the colors reverted, and new target windows and positions would be highlighted in red and yellow, respectively. In each condition, users were instructed to position each of the six windows in the space to the rest five positions (excluding the original position) twice, totaling 180 movement operations per user. The appearance order of the target windows and positions was randomized. 4.3.4 Task 4: Target Window Scaling In this task, participants were required to scale the target window highlighted in red with a dashed outline indicating the desired size. If the current size was within a 5% error range of the target size when the user completes the scaling operation, the current trial was considered completed. The original window size would be restored, and a new target window and target size would be generated. Each participant performed five scaling operations in both zoom-in (1.2 × original size) and zoom-out (0.8 × original size) modes on each window as suggested by Li et al. [40,41], totaling 180 operations per participant. The scaling factors were chosen to provide adequate fingertip travel distance for the palm-based gesture, while simultaneously avoiding significant visual clutter and inter-window occlusion. The appearance orders of the target windows and sizes were randomized. 4.4 Window Management Techniques In this study, participants were assigned assessment tasks utilizing three distinct techniques. Notably, Virtual Hand and Controller served as baseline methodologies, aligning with the default window management interactions of the Meta Horizon OS [40, 41]. A key methodological adjustment was made for a fair comparison. Unlike the standard inter- actions of Meta Horizon OS, which restricts manipulation to window borders to avoid conflicting with in-window content interactions, our baselines permitted interaction with the entire window surface. This decision equalized target acquisition difficulty with the conflict-free Handows technique. The functional outcomes of all window opera- tions were standardized across all three conditions. The operational procedures for each baseline technique are delineated as follows: 4.4.1 Virtual Hand (Ray-casting and Gesture) For the Virtual Hand technique, participants use a pinch gesture when the ray projected from the center of the palm intersects with the target window area to select it. Releasing the pinch confirms the selection. Similarly, closing a target window involves a pinch gesture when the ray aligns with the close button, and releasing the pinch confirms closure. To move a target window, participants utilize a pinch gesture when the ray intersects with the window area and then the selected window follows the hand’s movement. Releasing the pinch halts the window’s movement. Participants use a pinch gesture within the window’s border area to resize a target window. Dragging outward enlarges the window while dragging inward reduces its size. 4.4.2 Controller (Ray-casting and Button) For the Controller technique, participants interact with windows by positioning the ray projected from the front end of the controller onto the target window area to select it. Selection is initiated by pressing the trigger button, with confirmation upon trigger release. Similarly, closing a target window is executed by positioning the ray onto the close button and pressing the trigger button, confirming closure upon trigger release. To move a target window, participants position the ray onto the window area and hold the trigger button to enable the window to track the controller’s movement. Releasing the trigger button halts the window’s movement. Resizing operations are initiated by positioning the ray onto the window border and pressing the trigger button. Participants then drag outward to enlarge the window or inward to reduce its size, confirming the action upon trigger release. 4.5 Results We analyzed both objective and subjective measures across all tasks and conditions. Normality was assessed using the Shapiro–Wilk test. For normally distributed variables, we applied repeated-measures ANOVA (RM-ANOVA) with Tukey HSD post-hoc comparisons. For non- normally distributed variables, we used the Friedman test with pairwise Mann–Whitney U tests and Bonferroni correction. The significance level was set at p < .05. Descriptive statistics are summarized in Ta- ble 1. 4.5.1 Task 1: Target Window Selection Task Completion Time. A significant main effect was found (F(2,28) = 28.27, p < .001). Post-hoc tests showed that both HANDOWS and CONTROLLER were significantly faster than VIRTUAL HAND (p = .0008, p = .0001), with no difference between the former two (p = .804). Hand Movement Distance. Friedman test revealed a signifi- cant effect (χ2(2) = 19.60, p < .001). Post-hoc results indicated that HANDOWS and CONTROLLER required shorter distances than VIR- TUAL HAND (p= .0008, p< .0001). No difference was found between HANDOWS and CONTROLLER (p = .320). Head Rotation Angle. A significant difference was observed (F(2,28) = 58.83, p < .001). HANDOWS resulted in significantly smaller head rotation than both CONTROLLER and VIRTUAL HAND (both p < .0001). The difference between CONTROLLER and VIRTUAL HAND was not significant (p = .071). 4.5.2 Task 2: Target Window Closure Task Completion Time. A main effect was found (F(2,28) = 96.41, p < .001). HANDOWS was significantly faster than both CON- TROLLER (p = .009) and VIRTUAL HAND (p < .0001). CONTROLLER also outperformed VIRTUAL HAND (p < .0001). Hand Movement Distance. Significant differences were found (F(2,28) = 142.50, p< .001). HANDOWS and CONTROLLER required less hand movement than VIRTUAL HAND (p = .026, p = .0003). Head Rotation Angle. Friedman test revealed a significant effect (χ2(2) = 26.53, p < .001). HANDOWS elicited significantly smaller head rotation than both CONTROLLER and VIRTUAL HAND (both p < .0001). CONTROLLER also differed from VIRTUAL HAND (p = .0004). 4.5.3 Task 3: Target Window Positioning Task Completion Time. A significant main effect was found (F(2,28) = 28.70, p < .001). CONTROLLER was faster than both HANDOWS and VIRTUAL HAND (both p < .0001). No difference was observed between HANDOWS and VIRTUAL HAND (p = .732). Hand Movement Distance. The effect was significant (χ2(2) = 24.13, p < .001). HANDOWS outperformed both CONTROLLER (p = .002) and VIRTUAL HAND (p < .0001). CONTROLLER also outperformed VIRTUAL HAND (p = .0002). Head Rotation Angle. ANOVA revealed a significant effect (F(2,28) = 54.26, p < .001). HANDOWS involved less head rota- tion than both CONTROLLER and VIRTUAL HAND (both p < .0001), and CONTROLLER also differed from VIRTUAL HAND (p = .043). 4.5.4 Task 4: Target Window Scaling Task Completion Time. A significant effect was found (F(2,28) = 110.62, p < .001). HANDOWS was significantly faster than both CONTROLLER (p = .031) and VIRTUAL HAND (p < .0001). Hand Movement Distance. Differences were significant (F(2,28) = 56.52, p< .001). HANDOWS required less hand movement than both CONTROLLER (p = .0008) and VIRTUAL HAND (p < .0001). Head Rotation Angle. Friedman test showed a significant ef- fect (χ2(2) = 28.13, p < .001). HANDOWS resulted in significantly smaller head rotation compared to both CONTROLLER (p = .0002) and VIRTUAL HAND (p < .0001). Scaling Deviation. In Task 4, we additionally measured scaling deviation. A repeated-measures ANOVA revealed a significant main effect of interaction technique on deviation (F(2,28) = 41.60, p < .001). HANDOWS (M = 1.39%, SD = 0.22%) resulted in significantly more accurate scaling compared to both CONTROLLER (M = 2.02%, SD = 0.25%) and VIRTUAL HAND (M = 2.04%, SD = 0.24%), with both comparisons reaching significance (p < .0001). 4.5.5 Questionnaire Results User Experience (UEQ-S). Significant differences were found in Pragmatic Quality (χ2(2) = 17.16, p = .0002), Hedonic Qual- ity (χ2(2) = 10.56, p = .005), and Overall Score (χ2(2) = 14.88, p = .0006). Post-hoc comparisons showed that HANDOWS and CON- TROLLER significantly outperformed VIRTUAL HAND in Pragmatic Quality (p = .001, p = .0004, respectively). HANDOWS also exceeded both alternatives in Hedonic Quality (p < .0006 vs. CONTROLLER, Table 2: Questionnaire results showing the mean (SD) for each input technique across all measured dimensions, including user experience (UEQ-S), workload (NASA-TLX), fatigue (Borg 6-20), simulator sickness, system usability (SUS) and final preference. Measure HANDOWS VIRTUAL HAND CONTROLLER UEQ-S Pragmatic Quality 6.28 (0.75) 4.25 (1.42) 6.02 (0.69) Hedonic Quality 6.05 (0.67) 4.57 (1.17) 4.52 (1.46) Overall Score 6.04 (0.64) 4.41 (1.16) 5.40 (0.99) NASA-TLX Mental Demand 2.20 (0.77) 3.40 (1.59) 2.13 (1.46) Physical Demand 2.53 (1.19) 5.00 (1.31) 2.80 (1.66) Performance 1.80 (0.68) 3.60 (1.64) 2.00 (1.13) Effort 2.40 (1.16) 4.80 (1.82) 2.93 (1.80) Frustration 1.60 (0.63) 3.80 (1.86) 1.73 (0.88) Borg 6-20 11.20 (1.73) 15.20 (2.20) 10.27 (2.36) Simulator Sickness 3.67 (2.31) 7.53 (4.94) 3.13 (1.98) System Usability (SUS) 73.50 (17.02) 60.33 (15.31) 67.83 (13.75) p < .002 vs. VIRTUAL HAND). For Overall Score, only HANDOWS significantly outperformed VIRTUAL HAND (p = .001). Perceived Workload (NASA-TLX). Significant differences were observed in Mental Demand, Physical Demand, Performance, Ef- fort, and Frustration (p < .01 for all), but not in Temporal Demand (p = .216). VIRTUAL HAND consistently induced higher workload across all significant dimensions when compared to both HANDOWS and CONTROLLER (p < .05). Notably, HANDOWS also required signif- icantly less effort than CONTROLLER (p = .003). Perceived Fatigue (Borg Scale). Fatigue scores differed signif- icantly across techniques (χ2(2) = 16.93, p = .0002). Participants reported significantly higher fatigue with VIRTUAL HAND compared to both HANDOWS (p < .0001) and CONTROLLER (p = .0003). Simulator Sickness. A significant overall effect was observed (χ2(2) = 10.29, p = .006), but post-hoc comparisons did not yield significant pairwise differences after correction. System Usability (SUS). No significant differences were found across techniques (χ2(2) = 2.53, p = .282). All techniques received above-average SUS scores, suggesting generally acceptable levels of usability across conditions. Final Preference. Most participants (N = 10) ranked HANDOWS highest in performance, 5 favored CONTROLLER, and none preferred VIRTUAL HAND. 4.6 Summary of Results The results across both objective and subjective measures consistently indicate that Handows offers substantial advantages in immersive win- dow management. In particular, it significantly outperformed both Controller and Virtual Hand in window closure (Task 2) and scaling (Task 4), demonstrating faster task completion, reduced physical effort, and greater precision in scaling accuracy. For window selection (Task 1), Handows showed comparable performance to Controller and was notably faster than Virtual Hand. In contrast, for window positioning (Task 3), Controller achieved the best performance, while Handows did not show a statistically significant difference from Virtual Hand. The performance gap in Task 3 can be attributed to the nature of positioning tasks, which require continuous control and benefit from the precision and stability of dedicated ray-casting hardware. While the Controller condition offered reliable selection and tracking through button-based input, both Handows and Virtual Hand depended on hand gesture recognition, which may introduce latency and variability in tracking. Despite these limitations, Handows still achieved competitive performance without relying on external hardware—leveraging only passive haptics and familiar gesture paradigms. Moreover, the baseline conditions allowed unrestricted interaction with the full window sur- face, whereas Handows was intentionally designed around a compact, Fig. 6: Case study: Trip planning and budgeting using Handows, showcasing multitasking and dynamic window management in VR. ergonomic palm interface. These findings suggest that while Handows does not surpass the Controller in every metric, it delivers robust and efficient interaction in most tasks, thus partially supporting our H1. In terms of ergonomic efficiency, Handows consistently resulted in reduced head rotation and hand movement across all tasks. Subjective measures further reinforce its benefits: participants reported lower men- tal and physical workload, less fatigue, and greater overall satisfaction with Handows compared to both baselines, especially Virtual Hand. Although the SUS scores were similar across techniques, Handows was rated highest in both pragmatic and hedonic quality dimensions, and was preferred by a majority of participants. Taken together, these findings provide strong evidence in support of H2 and H3. 5 CASE STUDY: REALISTIC MULTITASKING SCENARIO To explore the applicability of Handows, we conducted a case study involving a realistic multitasking scenario. The goal was to examine how users employed Handows to perform continuous window manage- ment operations in the context of a structured, goal-driven workflow, simulating common productivity scenarios in VR. 5.1 Scenario and Method We developed a Trip Planning and Budgeting scenario in which partici- pants gathered, organized, and synthesized information across multiple spatial browser windows (see Figure 6). The task involved sequential steps such as reviewing emails, checking weather forecasts, booking flights and accommodations, exploring restaurants and attractions, and compiling the final itinerary. This scenario was designed to elicit con- tinuous use of all four core window operations (selection, closure, positioning, and scaling), which were previously evaluated in our user study, thus allowing us to assess the real-world fluidity and adaptability of Handows in a dynamic VR workflow. 8 right-handed participants (5 male, 3 female), aged 20–41 (M = 24.75, SD = 7.63) were recruited in this case study. Their familiarity with VR varied, with an average rating of (M = 2.75, SD = 1.75) on a 7-point Likert scale (1 = very unfamiliar, 7 = very familiar). None had taken part in the previous user study. The case study was conducted using the same apparatus as in the earlier tasks. After a brief training session with a blank layout to practice the core operations, participants completed the trip planning scenario using only Handows, which took approximately 10 minutes. Post-task feedback on ease of use and enjoyment was collected via two 7-point Likert-scale questionnaires (1 = strongly disagree, 7 = strongly agree), as well as semi-structured interviews. The interviews focused on workflow adaptability, visual attention patterns, and overall user experience. 5.2 Results We analyzed the case study data using an inductive thematic analy- sis [10], which revealed three major themes: (1) Workflow Adaptability and Efficiency, (2) Visual Attention Strategies, and (3) Perceived Us- ability and Suggestions for Improvement. We also report quantitative ratings on ease of use and enjoyment to complement the qualitative findings. 5.2.1 Theme 1: Workflow Adaptability and Efficiency All participants (N = 8) agreed that Handows fit naturally into the multi- tasking scenario. They described the system as “intuitive,” “convenient,” and easy to integrate into continuous window operations. Several par- ticipants noted that the design resembled familiar mobile interactions. For example, P1 shared: “It feels like managing windows on my phone, and it aligns with my operating habits,” while P7 commented: “The hand panel simplifies the process and fits my intuitive operations.” Par- ticipants also highlighted that the unified access to multiple operations enhanced efficiency. P3 mentioned: “Multi-operations reduce the time spent switching between tools, making interactions more efficient.” Others appreciated the reduction in spatial workload. As P4 explained, “It reduces the fatigue of large-scale spatial movements, and the oper- ation is easy to understand and use.” P5 echoed this sentiment: “The system covers all the window operations I wanted to perform in VR while reducing my movement range.” When comparing Handows to prior experience with VR controllers, three participants emphasized its flexibility and accessibility. For exam- ple, P6 noted: “The hand panel follows the user’s position, allowing manipulation from any angle, which makes it more fluid and accessi- ble.” 5.2.2 Theme 2: Visual Attention Strategies Participants reported a consistent gaze pattern during early interac- tion: they looked up to locate a window, looked down to execute the command using Handows, and finally looked up again to verify the result. This sequence was especially prominent in selection, closure, and positioning tasks. For the scaling operation, participants described a cyclical gaze behavior. They repeatedly looked up to estimate the desired size and down at the hand panel to adjust the scaling gesture. P6 explained, “I still have to look at the panel when scaling, especially to confirm my finger posture at the start.” With increased familiarity, all participants (N = 8) were able to execute simple operations like selection and closure without referring to the hand panel, indicating a low visual-cognitive load. However, more complex tasks such as positioning and scaling continued to require visual confirmation. P1, P3, P5, and P7 remarked that positioning gestures were difficult to execute blindly, as they required precise start and end finger placement. Similarly, P1, P5, and P6 emphasized the importance of finger alignment when performing the multi-finger scaling gesture. 5.2.3 Theme 3: Perceived Usability and Suggestions for Im- provement Participants generally found Handows easy to use and enjoyable, rating its ease of use at (M = 5.25, SD = 0.46) and enjoyment at (M = 6.63, SD = 0.52) on a 7-point Likert scale. The mobile-inspired interaction was frequently described as “intuitive” and “familiar” (P1: “It feels like managing windows on my phone.”; P7: “The panel fits my intuitive operations well.”). While basic operations were smooth, more complex gestures posed challenges. Scaling was noted as physically tiring due to palm instability and recognition issues, and swapping felt “less smooth, especially when multiple windows were involved” (P5). Some participants also wished for features like reopening closed windows (P3, P5) or more layout customization (P1, P4). Suggested improvements included scaling speed adjustment (P7), visual feedback for positioning (P4), and recovery options for closed windows (P8). These comments reflect a desire to maintain Handows’ simplicity while enhancing control and flexibility in complex scenarios. 6 DISCUSSION Our study demonstrates that Handows supports effective and efficient window management in VR, not only across isolated tasks but also within realistic multitasking workflows. By combining embodied input with familiar interaction metaphors, Handows bridges the gap between physical ergonomics and digital flexibility. In this section, we syn- thesize findings from both controlled experiments and the case study, discussing how Handows enables task integration, supports embodied interaction, and reveals opportunities for future refinement. 6.1 Task Integration and Multitasking Support One of the key contributions of Handows lies in its capacity to facilitate fluid task switching across multiple window operations. In the four-task study, participants consistently performed selection and closure more quickly and with less head and hand movement compared to mid-air and controller-based alternatives. Tasks involving positioning and scaling, although more demanding, achieved high precision and benefited from the physical anchoring of the palm interface. Notably, scaling deviation was lowest in Handows (1.39%), supporting its suitability for fine- grained control. The case study further validated Handows in an open-ended multi- tasking scenario. Participants described the system as “intuitive” and “mobile-like,” often drawing parallels with familiar smartphone work- flows (P1, P7). They organically developed spatial strategies (e.g., prioritizing central windows, offloading peripheral content, and se- quencing closure and scaling) to maintain clarity and reduce cognitive clutter. These behaviors suggest that Handows not only supports dis- crete interactions but also enables higher-level workflow structuring. Even in the absence of explicit instruction, users adapted their own heuristics for spatial layout and operational sequencing, reinforcing the system’s learnability and flexibility. 6.2 Embodied Design and Familiarity as Enablers The performance and subjective results across both studies point to three key design considerations underlying Handows’ success: body- centric interaction, spatial miniaturization, and interaction familiarity. First, anchoring window control on the non-dominant palm provides proprioceptive stability and passive haptic cues. Users rated the impact of passive haptic feedback from touching the hand during Handows operation on operational stability as high, with an average score of 5.93 out of 7 (SD = 0.88). This embodiment reduced reliance on external spatial reference points, allowing for more compact movements and lower head rotation across tasks. Participants in the case study also noted that “the hand panel follows the user’s position,” making it easier to operate from varying viewpoints (P6). Second, the spatial miniaturization of interaction surfaces helped consolidate functions into a single accessible region. This design choice eliminated the need for large mid-air motions, improving speed and reducing fatigue. While scaling and positioning required visual confirmation, selection and closure were often performed without visual dependence after minimal training. Third, the use of well-established mobile gestures (e.g., taps, swipes, pinches) allowed participants to transfer prior experience from 2D devices into a 3D context. This was supported by subjective ratings assessing the transferability of experience from traditional comput- ing devices, where a Friedman test revealed significant differences (χ2(2) = 20.63, p < .0001). Handows scored highest at 6.13 out of 7 (SD = 0.74), significantly surpassing Controller at 4.13 (SD = 1.73, p = 0.0005) and Virtual Hand at 2.80 (SD = 1.15, p < 0.0001). This reuse of motor schemas was evident in both studies and aligns with the high ratings of Ease of Use and enjoyment in the case study. As P7 summarized, “the [palm-based] panel fits my intuitive operations well,” highlighting how gesture familiarity directly enhanced usability. 6.3 Limitations and Future Work Despite these advantages, our findings also surface areas for improve- ment. Participants reported occasional inaccuracies with high-precision gestures, particularly scaling. We attribute these challenges to a combi- nation of user-perceived palm instability and the inherent limitations of current hardwares and hand tracking, which can struggle with the stable recognition of fine-grained movements and maintaining tracking at the periphery of camera views. We anticipate that newer headsets like the Meta Quest 3, equipped with depth sensors and passthrough tracking, will enhance tracking robustness and extend the effective interaction space, enabling more fluid and reliable window manage- ment. Beyond tracking, adapting the current Handows prototype into a full-featured window management system will require dynamic lay- out adjustments to accommodate increasing numbers of windows and offer greater configurability. Participants also suggested specific en- hancements to support complex workflows, such as options to reopen closed windows (P3, P5), improved layout customization (P1, P4), and recovery mechanisms for handling interaction errors (P8). Future studies could incorporate individual hand measurements to investigate how physical differences affect interaction comfort and stability, thereby informing more personalized interface designs. Intro- ducing the movement patterns of the non-dominant hand would offer a deeper understanding of user motor behavior and the ergonomic im- plications of window management interaction. Our case study served as an initial simulation of continuous window operations in a realistic scenario; future work may extend this by integrating interactions with window content and combining Handows with complementary input modalities. This would enable systematic future studies, including formal evaluations of gesture input accuracy and other quantitative measurements, enhancing both robustness and usability validation. Beyond these system-level improvements, another promising direc- tion lies in understanding how user interaction patterns evolve with increased familiarity. While selection and closure were often performed without hand-panel observation, positioning and scaling remained vi- sually anchored. Longitudinal studies could examine whether users develop at-a-glance strategies that rely more on spatial muscle memory and less on visual confirmation, thus potentially unlocking faster and more immersive workflows. Participants also proposed improvements to feedback and flow, for example, visual animations during position- ing (P4) and dynamic scaling rate control (P7), which could support smoother transitions and reduce operational friction. Finally, integrat- ing multimodal input (e.g., eye tracking for target locking or voice commands for window grouping) may further expand the bandwidth of palm-based interaction [18, 43, 53]. Finally, while our study focused on VR, the implementation of Handows should also generalize well to MR settings. Its compact form factor, embodied layout, and reliance on familiar gestures make it especially suitable for productivity-focused MR environments [26]. Future work should explore collaborative extensions, adaptive palm interfaces based on user context, and integration with virtual workspace platforms. 7 CONCLUSION In this paper, we have presented Handows, a palm-based window management system for virtual reality that integrates miniature spatial interfaces, body-centric interaction, and familiar gesture paradigms. Designed to reduce physical strain and improve workflow efficiency, Handows supports core window operations (i.e., selection, closure, positioning, and scaling) within the user’s proprioceptive space using smartphone-inspired gestures. Through a user study (N=15), we demonstrated that Handows outper- forms common VR interaction techniques in task efficiency, precision, and user satisfaction, while significantly reducing physical effort. A complementary case study further validated its adaptability in realistic multitasking scenarios, where users employed strategic layout behav- iors and reported high engagement. These findings underscore the value of embedding spatial interfaces into the body for fluid, low-effort VR interaction. In the future, we see potential for extending Handows with adaptive layouts, multimodal input, and support for more complex workflows. More broadly, this work highlights how transplanting familiar inter- action models onto embodied surfaces can support the design of VR systems that are both powerful and accessible. ACKNOWLEDGMENTS Xiang Li is supported by the China Scholarship Council (CSC) Inter- national Cambridge Scholarship (No. 202208320092). We thank all anonymous reviewers for their valuable feedback and our participants for their time and contributions. REFERENCES [1] A. AlJarrah and M. Shehab. Maintaining user interface integrity on android. In 2016 IEEE 40th Annual Computer Software and Applications Confer- ence (COMPSAC), vol. 1, pp. 449–458, 2016. doi: 10.1109/COMPSAC. 2016.150 1 [2] T. Azai, S. Ogawa, M. Otsuki, F. Shibata, and A. Kimura. Selection and manipulation methods for a menu widget on the human forearm. In Proceedings of the 2017 CHI Conference Extended Abstracts on Human Factors in Computing Systems, CHI EA ’17, 4 pages, p. 357–360. Asso- ciation for Computing Machinery, New York, NY, USA, 2017. doi: 10. 1145/3027063.3052959 2 [3] T. Azai, S. Ushiro, J. Li, M. Otsuki, F. Shibata, and A. Kimura. Tap-tap menu: body touching for virtual interactive menus. In Proceedings of the 24th ACM Symposium on Virtual Reality Software and Technology, VRST ’18, article no. 57, 2 pages. Association for Computing Machinery, New York, NY, USA, 2018. doi: 10.1145/3281505.3281561 2 [4] X. Bi and R. Balakrishnan. Comparing usage of a large high-resolution display to single or dual desktop displays for daily work. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI ’09, 10 pages, p. 1005–1014. Association for Computing Machinery, New York, NY, USA, 2009. doi: 10.1145/1518701.1518855 2, 3 [5] V. Biener, T. Gesslein, D. Schneider, F. Kawala, A. Otte, P. O. Kristensson, M. Pahud, E. Ofek, C. Campos, M. Kljun, et al. Povrpoint: Authoring presentations in mobile virtual reality. IEEE Transactions on Visualization and Computer Graphics, 28(5):2069–2079, 2022. 2 [6] V. Biener, S. Kalamkar, N. Nouri, E. Ofek, M. Pahud, J. J. Dudley, J. Hu, P. O. Kristensson, M. Weerasinghe, K. Č. Pucihar, et al. Quantifying the effects of working in vr for one week. IEEE Transactions on Visualization and Computer Graphics, 28(11):3810–3820, 2022. 1 [7] V. Biener, D. Schneider, T. Gesslein, A. Otte, B. Kuth, P. O. Kristensson, E. Ofek, M. Pahud, and J. Grubert. Breaking the screen: Interaction across touchscreen boundaries in virtual reality for mobile knowledge workers. IEEE Transactions on Visualization and Computer Graphics, 26(12):3490–3502, 2020. doi: 10.1109/TVCG.2020.3023567 1 [8] G. Borg. Borg’s Perceived Exertion And Pain Scales. 07 1998. 4 [9] D. A. Bowman, E. Kruijff, J. J. LaViola, and I. Poupyrev. 3D User Interfaces: Theory and Practice. Addison Wesley Longman Publishing Co., Inc., USA, 2004. 1 [10] V. Braun and V. Clarke. Using thematic analysis in psychology. Qualitative research in psychology, 3(2):77–101, 2006. 7 [11] J. Brooke. Sus: A quick and dirty usability scale. Usability Eval. Ind., 189, 11 1995. 4 [12] S. Burigat and L. Chittaro. Visualizing references to off-screen content on mobile devices: A comparison of arrows, wedge, and overview + detail. Interacting with Computers, 23:156–166, 03 2011. doi: 10.1016/j.intcom. 2011.02.005 2 [13] Canadian Centre for Occupational Health and Safety. Office ergonomics - positioning the monitor. https://www.ccohs.ca/oshanswers/ ergonomics/office/monitor_positioning.html, 11 2022. 3 [14] L.-W. Chan, H.-S. Kao, M. Y. Chen, M.-S. Lee, J. Hsu, and Y.-P. Hung. Touching the void: direct-touch interaction for intangible displays. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI ’10, 10 pages, p. 2625–2634. Association for Computing Machinery, New York, NY, USA, 2010. doi: 10.1145/1753326.1753725 2 [15] J. Chatain, D. M. Sisserman, L. Reichardt, V. Fayolle, M. Kapur, R. W. Sumner, F. Zünd, and A. H. Bermano. Digiglo: Exploring the palm as an input and display mechanism through digital gloves. In Proceedings of the Annual Symposium on Computer-Human Interaction in Play, CHI PLAY ’20, 12 pages, p. 374–385. Association for Computing Machinery, New York, NY, USA, 2020. doi: 10.1145/3410404.3414260 2 [16] Y. Cheng, Y. Yan, X. Yi, Y. Shi, and D. Lindlbauer. Semanticadapt: Optimization-based adaptation of mixed reality layouts leveraging virtual- physical semantic connections. In The 34th Annual ACM Symposium on User Interface Software and Technology, UIST ’21, 16 pages, p. 282–297. Association for Computing Machinery, New York, NY, USA, 2021. doi: 10.1145/3472749.3474750 2 [17] R. Dachselt and A. Hübner. Three-dimensional menus: A survey and taxonomy. Computers & Graphics, 31(1):53–65, 2007. 1 [18] S. Davari, F. Lu, and D. A. Bowman. Validating the benefits of glanceable and context-aware augmented reality for everyday information access tasks. In 2022 IEEE Conference on Virtual Reality and 3D User Interfaces (VR), pp. 436–444, 2022. doi: 10.1109/VR51125.2022.00063 8 [19] T. T. Elvins, D. R. Nadeau, and D. Kirsh. Worldlets—3d thumbnails for wayfinding in virtual environments. In Proceedings of the 10th Annual ACM Symposium on User Interface Software and Technology, UIST ’97, 10 pages, p. 21–30. Association for Computing Machinery, New York, NY, USA, 1997. doi: 10.1145/263407.263504 2 [20] B. Ens, J. D. Hincapié-Ramos, and P. Irani. Ethereal planes: a design framework for 2d information space in 3d mixed reality environments. In Proceedings of the 2nd ACM Symposium on Spatial User Interaction, SUI ’14, 11 pages, p. 2–12. Association for Computing Machinery, New York, NY, USA, 2014. doi: 10.1145/2659766.2659769 2 [21] B. M. Ens, R. Finnegan, and P. P. Irani. The personal cockpit: a spatial in- terface for effective task switching on head-worn displays. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI ’14, 10 pages, p. 3171–3180. Association for Computing Machinery, New York, NY, USA, 2014. doi: 10.1145/2556288.2557058 3 [22] C. M. Fang and C. Harrison. Retargeted self-haptics for increased im- mersion in vr without instrumentation. In The 34th Annual ACM Sympo- sium on User Interface Software and Technology, UIST ’21, 13 pages, p. 1109–1121. Association for Computing Machinery, New York, NY, USA, 2021. doi: 10.1145/3472749.3474810 2 [23] S. Feiner, B. MacIntyre, M. Haupt, and E. Solomon. Windows on the world: 2d windows for 3d augmented reality. In Proceedings of the 6th Annual ACM Symposium on User Interface Software and Technology, UIST ’93, 11 pages, p. 145–155. Association for Computing Machinery, New York, NY, USA, 1993. doi: 10.1145/168642.168657 1, 2 [24] L. Figueiredo, E. Rodrigues, J. Teixeira, and V. Teichrieb. A comparative evaluation of direct hand and wand interactions on consumer devices. Computers & Graphics, 77:108–121, 2018. doi: 10.1016/j.cag.2018.10. 006 1, 2 [25] G. W. Fitzmaurice. Situated information spaces and spatially aware palm- top computers. Commun. ACM, 36(7):39–49, 11 pages, jul 1993. doi: 10. 1145/159544.159566 2 [26] Z. Gao, X. Li, C. Liu, X. Wang, A. Wang, L. Yang, Y. Wang, P. Hui, and T. Braud. Vr prem+: An immersive pre-learning branching visualization system for museum tours. In Proceedings of the Eleventh International https://doi.org/10.1109/COMPSAC.2016.150 https://doi.org/10.1109/COMPSAC.2016.150 https://doi.org/10.1145/3027063.3052959 https://doi.org/10.1145/3027063.3052959 https://doi.org/10.1145/3281505.3281561 https://doi.org/10.1145/1518701.1518855 https://doi.org/10.1109/TVCG.2020.3023567 https://doi.org/10.1016/j.intcom.2011.02.005 https://doi.org/10.1016/j.intcom.2011.02.005 https://www.ccohs.ca/oshanswers/ergonomics/office/monitor_positioning.html https://www.ccohs.ca/oshanswers/ergonomics/office/monitor_positioning.html https://doi.org/10.1145/1753326.1753725 https://doi.org/10.1145/3410404.3414260 https://doi.org/10.1145/3472749.3474750 https://doi.org/10.1145/3472749.3474750 https://doi.org/10.1109/VR51125.2022.00063 https://doi.org/10.1145/263407.263504 https://doi.org/10.1145/2659766.2659769 https://doi.org/10.1145/2556288.2557058 https://doi.org/10.1145/3472749.3474810 https://doi.org/10.1145/168642.168657 https://doi.org/10.1016/j.cag.2018.10.006 https://doi.org/10.1016/j.cag.2018.10.006 https://doi.org/10.1145/159544.159566 https://doi.org/10.1145/159544.159566 Symposium of Chinese CHI, pp. 374–385, 2023. 3, 9 [27] T. Gonçalves, A. P. Afonso, M. B. Carmo, and P. P. de Matos. Comparison of off-screen visualization techniques with representation of relevance on mobile devices. In British Computer Society Conference on Human- Computer Interaction, 2013. 2 [28] J. Grubert, L. Witzani, A. Otte, T. Gesslein, M. Kranz, and P. O. Kristens- son. Text entry performance and situation awareness of a joint optical see-through head-mounted display and smartphone system. IEEE Trans- actions on Visualization and Computer Graphics, pp. 1–16, 2023. doi: 10. 1109/TVCG.2023.3309316 2 [29] S. Gustafson, C. Holz, and P. Baudisch. Imaginary phone: learning imaginary interfaces by transferring spatial memory from a familiar device. In Proceedings of the 24th Annual ACM Symposium on User Interface Software and Technology, UIST ’11, 10 pages, p. 283–292. Association for Computing Machinery, New York, NY, USA, 2011. doi: 10.1145/2047196 .2047233 2 [30] C. Harrison, S. Ramamurthy, and S. E. Hudson. On-body interaction: armed and dangerous. In Proceedings of the Sixth International Confer- ence on Tangible, Embedded and Embodied Interaction, TEI ’12, 8 pages, p. 69–76. Association for Computing Machinery, New York, NY, USA, 2012. doi: 10.1145/2148131.2148148 2 [31] C. Harrison, D. Tan, and D. Morris. Skinput: appropriating the body as an input surface. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI ’10, 10 pages, p. 453–462. Association for Computing Machinery, New York, NY, USA, 2010. doi: 10.1145/ 1753326.1753394 2 [32] S. G. Hart. Nasa-Task Load Index (NASA-TLX); 20 Years Later. Pro- ceedings of the Human Factors and Ergonomics Society Annual Meeting, 50(9):904–908, Oct. 2006. Publisher: SAGE Publications Inc. doi: 10. 1177/154193120605000909 4 [33] K. Hinckley, R. Pausch, J. C. Goble, and N. F. Kassell. A survey of design issues in spatial input. In Proceedings of the 7th Annual ACM Symposium on User Interface Software and Technology, UIST ’94, 10 pages, p. 213–222. Association for Computing Machinery, New York, NY, USA, 1994. doi: 10.1145/192426.192501 1, 2 [34] R. S. Kennedy, N. E. Lane, K. S. Berbaum, and L. Mg. Simulator sickness questionnaire: An enhanced method for quantifying simulator sickness. The International Journal of Aviation Psychology, 3:203–220, 1993. 4 [35] T. Kim, A. Karlson, A. Gupta, T. Grossman, J. Wu, P. Abtahi, C. Collins, M. Glueck, and H. B. Surale. Star: Smartphone-analogous typing in augmented reality. In Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology, UIST ’23, article no. 116, 13 pages. Association for Computing Machinery, New York, NY, USA, 2023. doi: 10.1145/3586183.3606803 2, 3 [36] J. H. Lee, S.-G. An, Y. Kim, and S.-H. Bae. Projective windows: Bringing windows in space to the fingertip. In Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems, CHI EA ’18, 1 pages, p. 1. Association for Computing Machinery, New York, NY, USA, 2018. doi: 10.1145/3170427.3186524 1, 2 [37] X. Li, W. He, S. Jin, J. Gugenheimer, P. Hui, H.-N. Liang, and P. O. Kristensson. Investigating Creation Perspectives and Icon Placement Preferences for On-Body Menus in Virtual Reality. Proc. ACM Hum.- Comput. Interact., 8(ISS), article no. 536, 19 pages, Oct. 2024. doi: 10. 1145/3698136 2 [38] X. Li, W. He, and P. O. Kristensson. Evaluating the usability of microgestures for text editing tasks in virtual reality. arXiv preprint arXiv:2504.04198, 2025. 2 [39] X. Li and P. O. Kristensson. Bend it, aim it, tap it: Designing an on- body disambiguation mechanism for curve selection in mixed reality. In Proceedings of the ACM Symposium on Spatial User Interaction, SUI ’25, pp. 1–12. ACM, New York, NY, USA, 2025. To appear. 2 [40] X. Li, J.-D. Wang, J. J. Dudley, and P. O. Kristensson. Swarm manipulation in virtual reality. In Proceedings of the 2023 ACM Symposium on Spatial User Interaction, pp. 1–11, 2023. 4, 5 [41] X. Li, J.-D. Wang, J. J. Dudley, and P. O. Kristensson. Swarm manipula- tion: An efficient and accurate technique for multi-object manipulation in virtual reality. Computers & Graphics, 125:104113, 2024. 4, 5 [42] D. Lindlbauer, A. M. Feit, and O. Hilliges. Context-aware online adapta- tion of mixed reality interfaces. In Proceedings of the 32nd Annual ACM Symposium on User Interface Software and Technology, UIST ’19, 14 pages, p. 147–160. Association for Computing Machinery, New York, NY, USA, 2019. doi: 10.1145/3332165.3347945 3 [43] F. Lu, S. Davari, L. Lisle, Y. Li, and D. A. Bowman. Glanceable ar: Evaluating information access methods for head-worn augmented reality. In 2020 IEEE Conference on Virtual Reality and 3D User Interfaces (VR), pp. 930–939, 2020. doi: 10.1109/VR46266.2020.00113 8 [44] Microsoft. Mixed reality design for ux elements: Holographic frame considerations. https://learn.microsoft.com/en-us/windows/ mixed-reality/design/comfort, 10 2021. 3 [45] F. F. Mueller, R. Patibanda, R. Byrne, Z. Li, Y. Wang, J. Andres, X. Li, J. Marquez, S. Greuter, J. Duckworth, and J. Marshall. Limited Control Over the Body as Intriguing Play Design Resource. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems, CHI ’21, pp. 1–16. Association for Computing Machinery, New York, NY, USA, May 2021. doi: 10.1145/3411764.3445744 2 [46] F. F. Mueller, N. Semertzidis, J. Andres, J. Marshall, S. Benford, X. Li, L. Matjeka, and Y. Mehta. Towards Understanding the Design of In- tertwined Human-computer Integrations. ACM Trans. Comput.-Hum. Interact., 30(5), article no. 73, 45 pages, sep 2023. doi: 10.1145/3590766 2 [47] B. Myers. A taxonomy of window manager user interfaces. IEEE Com- puter Graphics and Applications, 8(5):65–84, 1988. doi: 10.1109/38.7762 1 [48] R. Patibanda, C. Hill, A. Saini, X. Li, Y. Chen, A. Matviienko, J. Knibbe, E. van den Hoven, and F. F. Mueller. Auto-paizo games: Towards under- standing the design of games that aim to unify a player’s physical body and the virtual world. Proc. ACM Hum.-Comput. Interact., 7(CHI PLAY), article no. 408, 26 pages, oct 2023. doi: 10.1145/3611054 2 [49] R. Patibanda, A. Saini, N. Overdevest, M. F. Montoya, X. Li, Y. Chen, S. Nisal, J. Andres, J. Knibbe, E. van den Hoven, and F. F. Mueller. Fused spectatorship: Designing bodily experiences where spectators become players. Proc. ACM Hum.-Comput. Interact., 7(CHI PLAY), article no. 403, 34 pages, oct 2023. doi: 10.1145/3611049 2 [50] L. Pavanatto, J. Grubert, and D. A. Bowman. Spatial bar: Exploring window switching techniques for large virtual displays. In 2025 IEEE Conference Virtual Reality and 3D User Interfaces (VR), pp. 186–194. IEEE, 2025. 2 [51] L. Pavanatto, F. Lu, C. North, and D. A. Bowman. Multiple monitors or single canvas? evaluating window management and layout strategies on virtual displays. IEEE Transactions on Visualization and Computer Graphics, pp. 1–15, 2024. doi: 10.1109/TVCG.2024.3368930 2 [52] L. Pavanatto, C. North, D. A. Bowman, C. Badea, and R. Stoakley. Do we still need physical monitors? an evaluation of the usability of ar virtual monitors for productivity work. In 2021 IEEE Virtual Reality and 3D User Interfaces (VR), pp. 759–767, 2021. doi: 10.1109/VR50410.2021.00103 1, 3 [53] K. Pfeuffer, B. Mayer, D. Mardanbegi, and H. Gellersen. Gaze + pinch interaction in virtual reality. In Proceedings of the 5th Symposium on Spatial User Interaction, SUI ’17, 10 pages, p. 99–108. Association for Computing Machinery, New York, NY, USA, 2017. doi: 10.1145/3131277 .3132180 8 [54] J. S. Pierce, A. S. Forsberg, M. J. Conway, S. Hong, R. C. Zeleznik, and M. R. Mine. Image plane interaction techniques in 3d immersive environments. In Proceedings of the 1997 Symposium on Interactive 3D Graphics, I3D ’97, p. 39–ff. Association for Computing Machinery, New York, NY, USA, 1997. doi: 10.1145/253284.253303 1, 2 [55] J. Ren, Y. Weng, C. Zhou, C. Yu, and Y. Shi. Understanding window man- agement interactions in ar headset + smartphone interface. In Extended Abstracts of the 2020 CHI Conference on Human Factors in Comput- ing Systems, CHI EA ’20, 8 pages, p. 1–8. Association for Computing Machinery, New York, NY, USA, 2020. doi: 10.1145/3334480.3382812 2 [56] G. Robertson, M. van Dantzich, D. Robbins, M. Czerwinski, K. Hinckley, K. Risden, D. Thiel, and V. Gorokhovsky. The task gallery: a 3d window manager. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI ’00, 8 pages, p. 494–501. Association for Computing Machinery, New York, NY, USA, 2000. doi: 10.1145/332040. 332482 1 [57] C. Rooney and R. Ruddle. Improving window manipulation and content interaction on high-resolution, wall-sized displays. International Journal of Human-Computer Interaction, 28(7):423–432, 2012. 1 [58] R. W. Scheifler and J. Gettys. The x window system. ACM Trans. Graph., 5(2):79–109, 31 pages, Apr. 1986. doi: 10.1145/22949.24053 1 [59] M. Schrepp, J. Thomaschewski, and A. Hinderks. Construction of a benchmark for the user experience questionnaire (ueq). International Journal of Interactive Multimedia and Artificial Intelligence, 2017. 4 [60] L. Shupp, R. Ball, B. Yost, J. Booker, and C. North. Evaluation of viewport https://doi.org/10.1109/TVCG.2023.3309316 https://doi.org/10.1109/TVCG.2023.3309316 https://doi.org/10.1145/2047196.2047233 https://doi.org/10.1145/2047196.2047233 https://doi.org/10.1145/2148131.2148148 https://doi.org/10.1145/1753326.1753394 https://doi.org/10.1145/1753326.1753394 https://doi.org/10.1177/154193120605000909 https://doi.org/10.1177/154193120605000909 https://doi.org/10.1145/192426.192501 https://doi.org/10.1145/3586183.3606803 https://doi.org/10.1145/3170427.3186524 https://doi.org/10.1145/3698136 https://doi.org/10.1145/3698136 https://doi.org/10.1145/3332165.3347945 https://doi.org/10.1109/VR46266.2020.00113 https://learn.microsoft.com/en-us/windows/mixed-reality/design/comfort https://learn.microsoft.com/en-us/windows/mixed-reality/design/comfort https://doi.org/10.1145/3411764.3445744 https://doi.org/10.1145/3590766 https://doi.org/10.1109/38.7762 https://doi.org/10.1145/3611054 https://doi.org/10.1145/3611049 https://doi.org/10.1109/TVCG.2024.3368930 https://doi.org/10.1109/VR50410.2021.00103 https://doi.org/10.1145/3131277.3132180 https://doi.org/10.1145/3131277.3132180 https://doi.org/10.1145/253284.253303 https://doi.org/10.1145/3334480.3382812 https://doi.org/10.1145/332040.332482 https://doi.org/10.1145/332040.332482 https://doi.org/10.1145/22949.24053 size and curvature of large, high-resolution displays. In Proceedings of Graphics Interface 2006, GI ’06, 8 pages, p. 123–130. Canadian Informa- tion Processing Society, CAN, 2006. 2 [61] R. Stoakley, M. J. Conway, and R. Pausch. Virtual reality on a wim: interactive worlds in miniature. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI ’95, 8 pages, p. 265–272. ACM Press/Addison-Wesley Publishing Co., USA, 1995. doi: 10.1145/ 223904.223938 2 [62] C.-Y. Wang, W.-C. Chu, P.-T. Chiu, M.-C. Hsiu, Y.-H. Chiang, and M. Y. Chen. Palmtype: Using palms as keyboards for smart glasses. In Proceed- ings of the 17th International Conference on Human-Computer Interaction with Mobile Devices and Services, MobileHCI ’15, 8 pages, p. 153–160. Association for Computing Machinery, New York, NY, USA, 2015. doi: 10.1145/2785830.2785886 2 [63] C.-Y. Wang, M.-C. Hsiu, P.-T. Chiu, C.-H. Chang, L. Chan, B.-Y. Chen, and M. Y. Chen. Palmgesture: Using palms as gesture interfaces for eyes- free input. In Proceedings of the 17th International Conference on Human- Computer Interaction with Mobile Devices and Services, MobileHCI ’15, 10 pages, p. 217–226. Association for Computing Machinery, New York, NY, USA, 2015. doi: 10.1145/2785830.2785885 2 [64] E. Whitmire, M. Jain, D. Jain, G. Nelson, R. Karkar, S. Patel, and M. Goel. Digitouch: Reconfigurable thumb-to-finger input and text entry on head- mounted displays. Proc. ACM Interact. Mob. Wearable Ubiquitous Tech- nol., 1(3), article no. 113, 21 pages, sep 2017. doi: 10.1145/3130978 2 [65] D. Yu, Q. Zhou, J. Newn, T. Dingler, E. Velloso, and J. Goncalves. Fully- occluded target selection in virtual reality. IEEE Transactions on Vi- sualization and Computer Graphics, 26(12):3402–3413, 2020. doi: 10. 1109/TVCG.2020.3023606 2 https://doi.org/10.1145/223904.223938 https://doi.org/10.1145/223904.223938 https://doi.org/10.1145/2785830.2785886 https://doi.org/10.1145/2785830.2785886 https://doi.org/10.1145/2785830.2785885 https://doi.org/10.1145/3130978 https://doi.org/10.1109/TVCG.2020.3023606 https://doi.org/10.1109/TVCG.2020.3023606 Introduction Related Work Window Management in Virtual Environments Miniature Representations and Spatial Navigation On-Body Interaction and Passive Haptics Handows: A Window Management System for VR Spatial Layout Optimization Curved Layout Selection Palm-Based Interface and Interaction Design Features User Study Participants and Apparatus Method and Procedure Tasks and Measurement Task 1: Target Window Selection Task 2: Target Window Closure Task 3: Target Window Positioning Task 4: Target Window Scaling Window Management Techniques Virtual Hand (Ray-casting and Gesture) Controller (Ray-casting and Button) Results Task 1: Target Window Selection Task 2: Target Window Closure Task 3: Target Window Positioning Task 4: Target Window Scaling Questionnaire Results Summary of Results Case Study: Realistic Multitasking Scenario Scenario and Method Results Theme 1: Workflow Adaptability and Efficiency Theme 2: Visual Attention Strategies Theme 3: Perceived Usability and Suggestions for Improvement Discussion Task Integration and Multitasking Support Embodied Design and Familiarity as Enablers Limitations and Future Work Conclusion