|Abstract:||Recent years witnessed the emergence of more .multi-stream. Multimedia applications, such as multi-site distributed video conferencing, multi-perspective IPTV, and virtual touring of 3D digital worlds from multiple view points, etc. This trend is the result of greater availability of high performance computing devices, high speed Internet access and high quality video/audio capture devices.
In this work, we study the problem of visual space management in such applications. The key observation is that there are multiple visual streams that need to be presented to the users, while the visual space is limited both by availability of physical display devices and by user.s visual attention space. We argue that the visual space is a type of critical resource that requires careful management in similar way as other resources (such as CPU cycles and memory slots). Specifically, the unique challenges are how to model the visual streams and the viewers. preference among them, how to determine which of these visual streams should be presented to the viewer during the runtime of an application session, and how the output windows of these selected streams should be arranged on the limited display devices.
Many existing systems have explored the solution space from different perspectives. Some systems simply squeeze all available visual streams onto the display device; some systems let the viewers manually manage the screen layout for what they want to watch; some systems resort to professional human editors for help; some systems employ intelligent decision engines to automate the stream selection process by computers. Overall, none of the existing systems provide a solution that is customized to user interest, automatic to avoid unnecessary user input, and yields high visual quality screen output.
In this dissertation, we develop a 3-phase visual space management framework called MyView that makes the following three key contributions. First, a generic shot-based visual stream model and a score-based viewer interest model are designed to represent user interest, which lays the foundation for automatic visual space management decisions. Second, an automatic stream selection algorithm is developed that maintains a timed automaton to calculate which visual streams should be selected for display. As opposed to manually specified automaton used in related work, our approach automatically translates the viewer interest into a timed automaton and so adapts to changes in viewer interest dynamically during the runtime. Third, a screen layout calculation algorithm is designed to compute the screen layout for presenting the selected visual stream(s) on one or multiple display devices in a visually pleasing way that maximizes screen space utilization.
We have built a lecture recording and broadcasting system on top of the ConferenceXP platform to evaluate our solution. Results from computer simulation experiments and a mid-scale user study have confirmed the functional correctness and usability of the MyView framework.