Article

LADDER, a sketching language for user interface developers

Authors:

Randall DavisAuthors Info & Claims

SIGGRAPH '07: ACM SIGGRAPH 2007 courses

Pages 35 - es

https://doi.org/10.1145/1281500.1281546

Published: 05 August 2007 Publication History

Abstract

Sketch recognition systems are currently being developed for many domains, but can be time consuming to build if they are to handle the intricacies of each domain. In order to aid sketch-based user interface developers, we have developed tools to simplify the development of a new sketch recognition interface. We created LADDER, a language to describe how sketched diagrams in a domain are drawn, displayed, and edited. We then automatically transform LADDER structural descriptions into domain specific shape recognizers, editing recognizers, and shape exhibitors for use in conjunction with a domain independent sketch recognition system, creating a sketch recognition system for that domain. We have tested our framework by writing several domain descriptions and automatically generating a domain specific sketch recognition system from each description.

References

[1]

Alvarado C. A natural sketching environment: bringing the computer into early stages of mechanical design. Master's thesis, MIT, 2000.

[2]

Landay JA, Myers BA. Interactive sketching for the early stages of user interface design. In: Proceedings of CHI '95: Human Factors in Computing Systems, 1995. p. 43--50.

Digital Library

[3]

Stahovich T. Sketchit: a sketch interpretation tool for conceptual mechanism design. Technical Report, MIT AI Laboratory, 1996.

Digital Library

[4]

Hammond T, Davis R. Tahuti: a geometrical sketch recognition system for uml class diagrams, AAAI Spring Symposium on Sketch Understanding 2002; 59--68.

[5]

Damm CH, Hansen KM, Thomsen M. Tool support for cooperative object-oriented design: gesture based modeling on an electronic whiteboard. In: CHI 2000, CHI; 2000. p. 518--25.

Digital Library

[6]

Ideogramic, Ideogramic UML#8482;, Ideogramic ApS, Denmark, http://www.ideogramic.com/products/, 2001.

[7]

Lank E, Thorley JS, Chen SJ-S. An interactive system for recognizing hand drawn UML diagrams. In: Proceedings for CASCON 2000; 2000. p. 7.

Digital Library

[8]

Lin J, Newman MW, Hong JI, Landay J. DENIM: finding a tighter fit between tools and practice for web site design. In: CHI Letters: Human Factors in Computing Systems, CHI; 2000. p. 510--7.

Digital Library

[9]

Caetano A, Goulart N, Fonseca M, Jorge J. JavaSketchIt: issues in sketching the look of user interfaces, Sketch Understanding. Papers from the 2002 AAAI Spring Symposium.

[10]

Lecolinet E. Designing guis by sketch drawing and visual programming. In: Proceedings of the International Conference on Advanced Visual Interfaces (AVI 1998), AVI, New York: ACM Press, 1998. p. 274--6. URLciteseer. nj.nec.com/lecolinet98designing.html

Digital Library

[11]

Do EY-L. Vr sketchpad---create instant 3d worlds by sketching on a transparent window. In: de Vries B, van Leeuwen JP, Achten HH, editors. CAAD Futures 2001. 2001. p. 161--72.

[12]

Mahoney JV, Fromherz MPJ. Handling ambiguity in constraint-based recognition of stick figure sketches. SPIE Document Recognition and Retrieval IX Conference, San Jose, CA.

[13]

Pittman J, Smith I, Cohen P, Oviatt S, Yang T. Quickset: a multimodal interface for military simulations. Proceedings of the Sixth Conference on Computer-Generated Forces and Behavioral Representation, 1996. p. 217--24.

[14]

Hse H, Shilman M, Newton AR, Landay J. Sketch-based user interfaces for collaborative object-oriented modeling. Berkley CS260 Class Project (December 1999).

[15]

Zue, Glass, Conversational interfaces: advances and challenges. Proceedings of IEEE, 2000. p. 1166--80.

[16]

VoiceXML Forum, http://www.voicexml.org/specs/VoiceXML-100.pdf, Voice eXtensible Markup Language, 1st ed. (07 March 2000).

[17]

Hammond T, Davis R. LADDER: a language to describe drawing, display, and editing in sketch recognition. Proceedings of the 2003 International Joint Conference on Artificial Intelligence (IJCAI).

Digital Library

[18]

Hammond T, Davis R. Automatically transforming symbolic shape descriptions for use in sketch recognition. Proceedings of the 19th National Conference on Artificial Intelligence (AAAI-04).

[19]

Rubine D. Specifying gestures by example. Computer Graphics 1991;25(4):329--37.

Digital Library

[20]

Long AC. Quill: a gesture design tool for pen-based user interfaces, Eecs department, computer science division, UC Berkeley, Berkeley, CA, December 2001.

[21]

Sezgin TM. Feature point detection and curve approximation for early processing in sketch recognition. Master's thesis, Massachusetts Institute of Technology, June 2001.

[22]

Alvarado C, Davis R. Sketchread: a multi-domain sketch recognition engine. In: Proceedings of UIST '04, 2004. p. 23--32.

Digital Library

[23]

Friedman-Hill E, Jess, the Java expert system shell. http://herzberg.ca.sandia.gov/jess, 2001.

[24]

Stiny G, Gips J. Shape grammars and the generative specification of painting and sculpture. In: Freiman CV, editor. Information processing, vol. 71. Amsterdam: North-Holland, 1972. p. 1460--5.

[25]

Gips J. Computer implementation of shape grammars. NSF/MIT Workshop on Shape Computation.

[26]

Futrelle RP, Nikolakis N. Efficient analysis of complex diagrams using constraint-based parsing. In: ICDAR-95 (International Conference on Document Analysis and Recognition), Montreal, Canada; 1995. p. 782--90.

Digital Library

[27]

Bimber O, Encarnacao LM, Stork A. A multi-layered architecture for sketch-based interaction within virtual environments. Computer and Graphics (Special Issue on Calligraphic Interfaces: Towards a New Generation of Interactive Systems) 2000; 24(6): 851--67.

[28]

Mahoney JV, Fromherz MPJ. Three main concerns in sketch recognition and an approach to addressing them. In: Sketch Understanding, Papers from the 2002 AAAI Spring Symposium. Stanford, CA: AAAI Press, 2002. p. 105--12.

[29]

Caetano A, Goulart N, Fonseca M, Jorge J. Sketching user interfaces with visual patterns. Proceedings of the First Ibero-American Symposium in Computer Graphics (SIACG02), 2002. p. 271--9.

[30]

Shilman M, Pasula H, Russell S, Newton R. Statistical visual language models for ink parsing. In: Sketch Understanding, Papers from the 2002 AAAI Spring Symposium. Stanford, CA: AAAI Press, 2002. p. 126--32.

[31]

Lee S-W. Recognizing circuit symbols with attributed graph matching. In: Baird H, Bunke H, Yamamoto K, editors. Structured document image analysis, 1992. p. 340--58.

[32]

Keating JP, Mason RL. Some practical aspects of covariance estimation. Pattern Recognition Letters 1985; 3(5):295--350.

[33]

Calhoun C, Stahovich TF, Kurtoglu T, Kara LB. Recognizing multi-stroke symbols. Sketch Understanding. Papers from the 2002 AAAI Spring Symposium, 2002. p. 15--23.

[34]

Gross MD, Do EY-L. Demonstrating the electronic cocktail napkin: a paper-like interface for early design. 'Common Ground' appeared in ACM Conference on Human Factors in Computing (CHI), 1996. p. 5--6.

Digital Library

[35]

Jacob RJK, Deligiannidis L, Morrison S. A software model and specification language for non-WIMP = user interfaces. ACM Transactions on Computer-Human Interaction 1999;6(1):1--46 URLciteseer.nj.nec.com/jacob99software.html.

Digital Library

[36]

Costagliola G, Tortora G, Orefice S, Lucia D. Automatic generation of visual programming environments. IEEE Computer 1995; 56--65.

Digital Library

[37]

Veselova O. Perceptually based learning of shape descriptions. Master's thesis, Massachusetts Institute of Technology, Cambridge, MA, 2003.

[38]

Hammond T, Davis R. Shady: a shape description debugger for use in sketch recognition. AAAI Fall Symposium on Making Pen-Based Interaction Intelligent and Natural.

Cited By

Vaithilingam PArawjo IGlassman E(2024)Imagining a Future of Designing with AI: Dynamic Grounding, Constructive Negotiation, and Sustainable MotivationProceedings of the 2024 ACM Designing Interactive Systems Conference10.1145/3643834.3661525(289-300)Online publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1145/3643834.3661525
AlMughrabi AHiary H(2024)Hand-drawn Electric Circuit Diagrams Recognition using Deep Learning2024 28th International Conference on Information Technology (IT)10.1109/IT61232.2024.10475731(1-4)Online publication date: 21-Feb-2024
https://doi.org/10.1109/IT61232.2024.10475731
Chen KForbus KSrinivasan BChhaya NUsher MElkind E(2023)Sketch recognition via part-based hierarchical analogical learningProceedings of the Thirty-Second International Joint Conference on Artificial Intelligence10.24963/ijcai.2023/331(2967-2974)Online publication date: 19-Aug-2023
https://dl.acm.org/doi/10.24963/ijcai.2023/331
Show More Cited By

LADDER, a sketching language for user interface developers
1. Computing methodologies
  1. Computer graphics

Recommendations

LADDER, a sketching language for user interface developers

Sketch recognition systems are currently being developed for many domains, but can be time consuming to build if they are to handle the intricacies of each domain. In order to aid sketch-based user interface developers, we have developed tools to ...
Sketching reality: Realistic interpretation of architectural designs

In this article, we introduce sketching reality, the process of converting a freehand sketch into a realistic-looking model. We apply this concept to architectural designs. As the sketch is being drawn, our system periodically interprets its 2.5D-...
Sketching User Experiences: The Hands-on Course
CHI EA '15: Proceedings of the 33rd Annual ACM Conference Extended Abstracts on Human Factors in Computing Systems

When designing novel user interfaces, paper-pencil sketches can support the design thinking process and are valuable for communicating design ideas to others. This hands-on course will demonstrate how to integrate sketching into researchers' and ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

SIGGRAPH '07: ACM SIGGRAPH 2007 courses

August 2007

6166 pages

ISBN:9781450318235

DOI:10.1145/1281500

Conference Chairs:
Sara McMains
University of California, Berkeley
,
Peter-Pike Sloan
Microsoft Corporation

Copyright © 2007 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGGRAPH: ACM Special Interest Group on Computer Graphics and Interactive Techniques

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 05 August 2007

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Article

Conference

SIGGRAPH07

Sponsor:

SIGGRAPH

SIGGRAPH07: Special Interest Group on Computer Graphics and Interactive Techniques Conference

August 5 - 9, 2007

California, San Diego

Acceptance Rates

Overall Acceptance Rate 1,822 of 8,601 submissions, 21%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

22
Total Citations
View Citations
194
Total Downloads

Downloads (Last 12 months)29
Downloads (Last 6 weeks)4

Reflects downloads up to 21 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Vaithilingam PArawjo IGlassman E(2024)Imagining a Future of Designing with AI: Dynamic Grounding, Constructive Negotiation, and Sustainable MotivationProceedings of the 2024 ACM Designing Interactive Systems Conference10.1145/3643834.3661525(289-300)Online publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1145/3643834.3661525
AlMughrabi AHiary H(2024)Hand-drawn Electric Circuit Diagrams Recognition using Deep Learning2024 28th International Conference on Information Technology (IT)10.1109/IT61232.2024.10475731(1-4)Online publication date: 21-Feb-2024
https://doi.org/10.1109/IT61232.2024.10475731
Chen KForbus KSrinivasan BChhaya NUsher MElkind E(2023)Sketch recognition via part-based hierarchical analogical learningProceedings of the Thirty-Second International Joint Conference on Artificial Intelligence10.24963/ijcai.2023/331(2967-2974)Online publication date: 19-Aug-2023
https://dl.acm.org/doi/10.24963/ijcai.2023/331
Taele PBlagojevic RHammond TRay SCherian JKoh J(2023)SketchRec 2023: 3rd Workshop on Sketch RecognitionCompanion Proceedings of the 28th International Conference on Intelligent User Interfaces10.1145/3581754.3584184(1-1)Online publication date: 27-Mar-2023
https://dl.acm.org/doi/10.1145/3581754.3584184
Arawjo IDeArmas ARoberts MBasu SParikh T(2022)Notational Programming for Notebook Environments: A Case Study with Quantum CircuitsProceedings of the 35th Annual ACM Symposium on User Interface Software and Technology10.1145/3526113.3545619(1-20)Online publication date: 29-Oct-2022
https://dl.acm.org/doi/10.1145/3526113.3545619
Xu PHuang YYuan TXiang THospedales TSong YWang L(2021)On Learning Semantic Representations for Large-Scale Abstract SketchesIEEE Transactions on Circuits and Systems for Video Technology10.1109/TCSVT.2020.304158631:9(3366-3379)Online publication date: Sep-2021
https://doi.org/10.1109/TCSVT.2020.3041586
BRIOLA DMASCARDI VGIOSEFFI M(2020)OntoScene, A Logic-Based Scene Interpreter: Implementation and Application in the Rock Art DomainTheory and Practice of Logic Programming10.1017/S147106841900046220:4(456-511)Online publication date: 15-Jan-2020
https://doi.org/10.1017/S1471068419000462
Gupta MMehndiratta PBhardwaj A(2019)Object Recognition in Hand Drawn Images Using Machine Ensembling Techniques and Smote SamplingInformation, Communication and Computing Technology10.1007/978-981-15-1384-8_19(228-239)Online publication date: 13-Nov-2019
https://doi.org/10.1007/978-981-15-1384-8_19
Kang BLaViola Jr. JWisniewski PMark GFussell SLampe Cschraefel mHourcade JAppert CWigdor D(2017)Structured Input Improves Usability and Precision for Solving Geometry-based Algebraic ProblemsProceedings of the 2017 CHI Conference on Human Factors in Computing Systems10.1145/3025453.3025468(4692-4702)Online publication date: 2-May-2017
https://dl.acm.org/doi/10.1145/3025453.3025468
Polsley SRay JHammond T(2017)SketchSeeker: Finding Similar SketchesIEEE Transactions on Human-Machine Systems10.1109/THMS.2017.264968447:2(194-205)Online publication date: Apr-2017
https://doi.org/10.1109/THMS.2017.2649684
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents