top of page
This site was designed with the
.com
website builder. Create your website today.Start Now

Externalize iPad Touch Screen With Physical Buttons

Project Idea

​Control iPad software functions using physical interaction modalities other than touch gestures

 

Motivation

  1. iPad screen is limited in space.

  2. iPad touch interaction modality is not suitable for all kind of applications such as games.

  3. iPad smooth touch surface does not provide vision disabled users with tangible interaction mechanism.

  4. Users need to interact with computing devices in both physical and tangible modes [1].

 

Objectives

  • Externalize iPad touch screen with physical push buttons

  • Enable unsighted users with an alternative interaction technique; rather than gestures

 

System

System consists of three main modules:

  1. PDF rendering software that is written in Objective C, called VT-Reader. User can navigate through a pdf document from page to page through two soft buttons: “next” and “previous”.

  2. Electronic circuit consists of Arduino Uno, two push buttons, LED, and three resistors.

  3. Serial communication between the microcontroller and the iPad; so that the two physical buttons can control VT-Reader and implement the functions of the soft “next” and “previous” buttons. Serial communication is implemented on two sides; one is written in Arduino sketch, and the other is written in XCode.

Hardware Components

 

Challenges and Notes

  1. P4 adapter kit P4b needs to be installed and soldered according to the diagram shown in fig1.a. Its final view should look like fig 1.b

  2. Debugging communication between Arduino and iPad is extremely difficult. This is because there is no way available to see the data transfer between the two of them. Neither Serial monitor in Arduino nor the Simulator in XCode gives you this facility.

  3. The Redpark Serial Cable for iOS is the only cable approved by Apple for connecting iOS device to almost anything using serial communication. However, according to Apple policy you still cannot upload your software to AppStore.

  4. Connecting iOS to sensors is very recent field, so there exist very few resources and references. However, a useful book is just released in September 2011 that is addressing this point: “iOS Sensor Apps with Arduino” http://shop.oreilly.com/product/0636920021179.do

  5. Redpark serial cable is programmable only through the SDK associated with the cable. It is a compiled library that you can override only certain methods of it. Not having the library as open source code puts real limitations on your developing and applications.

  6. In addition to importing Redpard library to your XCode project, you must import Apple Accessory framework.

  7. Each time you connect Redpark cable to iPad, you will see an error message “This accessory requires an application”. To avoid this message, you should declare support for the serial cable in “Supporting Files” group in Xcode.

 

References

[1] Rob Nero. 2010. Stacked User Inputs. Master’s Thesis- Malmö University. (http://stackedui.com/)

 

bottom of page