Overview
Traditional TV remote is not suitable for interacting with many smart TV applications, particularly when typing text is necessary, such as for searching, browsing, and social applications. On the other hand, the increasinglypervasive mobile devices are emerging as an interesting platform for TV interactions given their intuitive input and output modalities. Second-Screen Smart TV ( S3TV ) is an Android application for personal mobile devices. It focus on how to maintain the user's mental models by separation of content consumption and remote control interfaces, how to reduce visual attention shifting by leveraging eyes-free gesture controls, and how to design crossscreen UI elements by considering the user's operation context. We conducted comprehensive user studies, subjective usability analysis, and objective interaction analysis. The interaction analysis confirmed that S3TV was highly effective and touchscreen gesture controls were preferred over hard system keys. Overall the participants reported that they felt very satisfed and would like to continue to use S3TV..
Design Principles
Principle 1: S3TV app should have separate "content" and "control" modes, so the user can maintain a clear mental model that the smartphone acts as a content consumption device (e.g. description of the show or other related content) and a remote control (e.g. fast forward/rewind, pause/resume, and other video-related control) under different modes.
Principle 2: The control interface of the S3TV app should promote gesture-based control actions for eyes-free interaction. Nowadays consumers have already got used to touchscreen gestures such as tapping, swiping, and dragging on modern smartphones and tablets. These gestures do not consume significant visual attention so the user can focus on the TV shows.
Principle 3: The cross-screen UI element must be designed appropriately to re ect the user's operation context, as the user switches visual attention between TV and mobile devices. For instance, the gesture control of the video playback should have feedback on the TV and the gesture should be recognized anywhere on the mobile screen without looking at the mobile device. The video play button should have \Play on TV" or similar label so the user understands where the video will be shown.
User Interface
In the content mode:
Figure 1: User Interface
Simplified State machine diagram
In the content mode, there are five different screens, S0, S1, S2, S3, and S4 as listed in Figure. S0 is a welcome screen. S1, S2 and S3 are list screens. In a local environment, S1 is a video category list and S2 is a video list which belongs to the selected video category. On S1 and S2 we can search for a certain video, and it enters into the screen S3 that is a list screen of search results. The description and other information about a particular video is shown on S4. On the content screens, S3TV can enter into control mode by pressing the "Play on TV" button on S4 or the "Ctrl" button on the top right of S2 and S3 screens. In the control mode, S5 is the only screen, and can go back to one of the content screens with the BACK key.
Figure 2: State machine diagram
Tool Download
Video Playback Gesture-based Operations
1) Double tap for pause and play.
2) Drag to a particular video position.
3) Swipe left or right to forward or rewind.
4) Swipe up or down for volume adjustment.
5) V gesture for taking screenshot.
6) Long press for menu of additional operations.
7) Shake the device to change video.
8) Back key to exit control screen.
9) Display incoming calls on the TV.
User Study and Experience Evaluation
Subjective Usability Evaluation: Sus-a quick and dirty usability scale.
Objective Usability Evaluation: A toolkit for usability testing of mobile applications.
More Detail
S3TV Report
Contacts
lzwang@nju.edu.cn