S3TV: Second Screen Smart TV Application for Personal Mobile Devices
Introduction Download User Study



Traditional TV remote is not suitable for interacting with many smart TV applications, particularly when typing text is necessary, such as for searching, browsing, and social applications. On the other hand, the increasinglypervasive mobile devices are emerging as an interesting platform for TV interactions given their intuitive input and output modalities. Second-Screen Smart TV ( S3TV ) is an Android application for personal mobile devices. It focus on how to maintain the user's mental models by separation of content consumption and remote control interfaces, how to reduce visual attention shifting by leveraging eyes-free gesture controls, and how to design crossscreen UI elements by considering the user's operation context. We conducted comprehensive user studies, subjective usability analysis, and objective interaction analysis. The interaction analysis confirmed that S3TV was highly effective and touchscreen gesture controls were preferred over hard system keys. Overall the participants reported that they felt very satisfed and would like to continue to use S3TV..

Design Principles

Principle 1: S3TV app should have separate "content" and "control" modes, so the user can maintain a clear mental model that the smartphone acts as a content consumption device (e.g. description of the show or other related content) and a remote control (e.g. fast forward/rewind, pause/resume, and other video-related control) under different modes.

Principle 2: The control interface of the S3TV app should promote gesture-based control actions for eyes-free interaction. Nowadays consumers have already got used to touchscreen gestures such as tapping, swiping, and dragging on modern smartphones and tablets. These gestures do not consume significant visual attention so the user can focus on the TV shows.

Principle 3: The cross-screen UI element must be designed appropriately to re ect the user's operation context, as the user switches visual attention between TV and mobile devices. For instance, the gesture control of the video playback should have feedback on the TV and the gesture should be recognized anywhere on the mobile screen without looking at the mobile device. The video play button should have \Play on TV" or similar label so the user understands where the video will be shown.

User Interface

In the content mode:


Figure 1: User Interface

Simplified State machine diagram

In the content mode, there are five different screens, S0, S1, S2, S3, and S4 as listed in Figure. S0 is a welcome screen. S1, S2 and S3 are list screens. In a local environment, S1 is a video category list and S2 is a video list which belongs to the selected video category. On S1 and S2 we can search for a certain video, and it enters into the screen S3 that is a list screen of search results. The description and other information about a particular video is shown on S4. On the content screens, S3TV can enter into control mode by pressing the "Play on TV" button on S4 or the "Ctrl" button on the top right of S2 and S3 screens. In the control mode, S5 is the only screen, and can go back to one of the content screens with the BACK key.


Figure 2: State machine diagram

Tool Download

an Android-based S3TV app

Video Playback Gesture-based Operations

1) Double tap for pause and play.

2) Drag to a particular video position.

3) Swipe left or right to forward or rewind.

4) Swipe up or down for volume adjustment.

5) V gesture for taking screenshot.

6) Long press for menu of additional operations.

7) Shake the device to change video.

8) Back key to exit control screen.

9) Display incoming calls on the TV.

User Study and Experience Evaluation

Subjective Usability Evaluation: Sus-a quick and dirty usability scale.

Objective Usability Evaluation: A toolkit for usability testing of mobile applications.


pre-testing questionnaire.

post-testing questionnaire.

statistical data.

user task sequence(SQL)

More Detail

S3TV Report





©2011 SEG CS NJU IntroductionDownload