Figure 2-2 - uploaded by Daniel Halbert
Content may be subject to copyright.
Source publication
Programming by example is a way of programming a software system in its own user interface. The user of the system writes a program by giving an example of what the program should do. The system records the sequence of actions, and can perform it again. Programming by example allows a user to create programs without doing conventional programming....
Citations
... [54,55], MIT App Inventor 2 [56,57], Scratch 3.0 [43,45,58,59], etc., are widely used in primary school courses. These programming platforms require little training, and people can develop their programs, games, and animated environments without having to learn the complex code structures of traditional programming languages [45,[59][60][61][62][63]. ...
The development of creativity plays a decisive role in the future development of human life as it stimulates divergent thinking and grants the critical ability to innovate and solve problems. Therefore, the question of how to encourage students’ creativity has also attracted the attention of related research in various fields. Our study used the Scratch visual programming tool to allow students to create their own digital games. We used different technological devices as external stimuli during the research process to stimulate students’ creative ideas. We provided four control devices: a keyboard and mouse, PicoBoard, a 65-inch touch screen, and a Wii remote control for 92 fifth-grade students in four S-shaped classes. After a 12-week experimental process, students designed their original Scratch game. The results of the study found that differences in device properties correspond to different items of creativity: devices highly relevant to students’ life experiences improve the flexibility and elaboration of creativity. Innovative controls promote creative originality. The freedom of device control can increase the fluency of creativity. Therefore, providing control devices with different properties will allow teachers to establish learning environments that foster creativity. Finally, we speculate on the impact of other control devices on creativity based on the research results for future reference.
... Scratch [21], LOGO [22], and Alice [23] are examples of visual programming languages that are widely used in programming education. Exemplary programming [24] is a programming approach that aims to program by illustrating data movement to the computer. EUD environments that incorporate live programming have been proposed. ...
Programming is becoming a new literacy, and programming education is being implemented in various settings. One of the issues faced by novice programmers is a lack of problem-solving skills. They have difficulty planning abstract solutions in the program and expressing solutions as a program. This study proposes a programming learning system that supports novice programmers’ thinking processes when solving programming tasks. The system presents candidates for solutions in natural language and allows the user to solve the problem by choosing from a limited number of options. The method allows users to repeat quick trial-and-error solutions without being interrupted by coding-induced problems. We implemented the proposed system as a web application for use on a smartphone and applied it in a university’s introductory programming class. After applying the proposed system to 37 students for four weeks, we found a correlation between the number of times a user selected an option using the proposed system and the class task score.We confirmed the possibility that using the proposed system can improve problem-solving skills for programming tasks.
... One is Synthesize, Execute, Debug [28], a framework that attempts to bridge the "last mile" gap by introducing program repair into the program synthesis algorithm. A programming task is specified using both a natural language description and a set of input/output (I/O) pairs demonstrating what output is expected of the program, thereby combining text to code [29] and programming by example [30,31] paradigms typical for competitive programming [32]. Synthesize, Execute, Debug creates a first draft program using a generative model, compiles and executes it with given input examples. ...
Current approaches to program synthesis with Large Language Models (LLMs) exhibit a "near miss syndrome": they tend to generate programs that semantically resemble the correct answer (as measured by text similarity metrics or human evaluation), but achieve a low or even zero accuracy as measured by unit tests due to small imperfections, such as the wrong input or output format. This calls for an approach known as Synthesize, Execute, Debug (SED), whereby a draft of the solution is generated first, followed by a program repair phase addressing the failed tests. To effectively apply this approach to instruction-driven LLMs, one needs to determine which prompts perform best as instructions for LLMs, as well as strike a balance between repairing unsuccessful programs and replacing them with newly generated ones. We explore these trade-offs empirically, comparing replace-focused, repair-focused, and hybrid debug strategies, as well as different template-based and model-based prompt-generation techniques. We use OpenAI Codex as the LLM and Program Synthesis Benchmark 2 as a database of problem descriptions and tests for evaluation. The resulting framework outperforms both conventional usage of Codex without the repair phase and traditional genetic programming approaches.
... Programmers tried to improve the user interface but not always the efforts in this direction were successful. This condition led to research aimed at using graphics to facilitate programming skills, leading to the birth of Visual Programming (VP) (Halbert, 1984). By eliminating syntax, the graphical method focused on workflow, making visual programming an efficient tool even for skilled programmers. ...
... (Connection Manager, 1988). Source: Halbert, 1984. In the 2000s there was a new success in parametric design with a subsequent spread of programming tools (ex. ...
The aim of the thesis is the definition of a parametric modelling methodology that allows, in a short time and at a sustainable cost, the digital acquisition, modelling and analysis of urban aggregates with the aim of facilitating seismic vulnerability mapping actions in historic centres. The research involves the use of direct data (site surveys) and derived data (available geodata) for the realisation of a parametric City Information Model (CIM). Grasshopper, a Visual Programming Language (VPL) within Rhinoceros, already widely used in the scientific community, was chosen as the working environment for the parametric computational design.
The methodology consists of several progressive steps that can be pursued through two innovative methods (Survey-to-CIM and Scan-to-CIM) defined and developed in this thesis. The choice of which method to adopt depends on the availability of project resources and the type of urban centre being studied. The first method, Survey-to-CIM, is intended to be a low-cost solution that integrates different data and acquisition techniques (direct survey, photo-rectification, expeditious 360 photogrammetry, Street-Level imagery, etc.) within a parametric and responsive flow useful for smaller urban centres and without the need for special professional skills. The second method, Scan-to- CIM, is developed to automate the cognitive operations of interpretation and input of survey data performed by the surveyor. This task is carried out by an Artificial Intelligence system that uses Machine Learning techniques (in particular Random Forest) to identify remarkable geometric-architectural features (openings, walls, stringcourses) within point clouds obtained through digital surveying. Both methods lead to the definition of a parametric CIM system developed in VPL.
The generated CIM model is equivalent to the definition of a semantic 3D City Model since all generated geometries, from the envelope to the interior, adhere to a semantic structure that defines relationships and dependencies. Since there are no guidelines in literature regarding the semantic structuring of city models in a parametric environment, an innovative format defined CityGH is proposed in this thesis. Based on the CityJSON standard, it proposes the same semantic structure of CityJSON but adapting it to the Data Structure of Grasshopper (Data Trees). This format is therefore proposed as an interchange format within parametric models to facilitate their dissemination and application. Thanks to its semantic structure, the parametric CIM model is capable of storing attributes and metadata that are fundamental steps in facilitating the beginning of the FEM analysis activities.
The parametric CIM model defined so far also allows the extraction of structural geometric models, both NURBS and MESH, necessary for the execution of FEM analyses. In particular, a workflow was developed in the thesis that allows FEM analysis both within the VPL environment (Karamba3D, Alpaca4D - OpenSees plugins) and in external software dedicated to structural analysis (SAP2000, FEM-Design).
... Scratch [94], Bloqqi [43], EUD-MARS [5], App Inventor [129], AppSheet [46] are such "low-code/no-code" application development tools that offer visual drag-and-drop facilities. Similarly, there are several other research areas within the domains of HCI [102] and Software engineering, such as Visual Programming [31], Programming by example [49], End users programming [75], domain specific language [73,124], trigger action programming [123] that aim to enhance the technologies underlying low-code software development. Thus, gaining a better knowledge of the problems associated with low-code platforms through developer discussion would be extremely beneficial for further improving these studies. ...
Low-code software development (LCSD) is an emerging approach to democratize application development for software practitioners from diverse backgrounds. LCSD platforms promote rapid application development with a drag-and-drop interface and minimal programming by hand. As it is a relatively new paradigm, it is vital to study developers' difficulties when adopting LCSD platforms. Software engineers frequently use the online developer forum Stack Overflow (SO) to seek assistance with technical issues. We observe a growing body of LCSD-related posts in SO. This paper presents an empirical study of around 33K SO posts containing discussions of 38 popular LCSD platforms. We use Topic Modeling to determine the topics discussed in those posts. Additionally, we examine how these topics are spread across the various phases of the agile software development life cycle (SDLC) and which part of LCSD is the most popular and challenging. Our study offers several interesting findings. First, we find 40 LCSD topics that we group into five categories: Application Customization, Database, and File Management, Platform Adoption, Platform Maintenance, and Third-party API Integration. Second, while the Application Customization (30\%) and Data Storage (25\%) \rev{topic} categories are the most common, inquiries relating to several other categories (e.g., the Platform Adoption \rev{topic} category) have gained considerable attention in recent years. Third, all topic categories are evolving rapidly, especially during the Covid-19 pandemic. The findings of this study have implications for all three LCSD stakeholders: LCSD platform vendors, LCSD developers/practitioners, Researchers, and Educators. Researchers and LCSD platform vendors can collaborate to improve different aspects of LCSD, such as better tutorial-based documentation, testing, and DevOps support.
... El paradigma de programación visual ha existido durante varias décadas; desde principios de la década de 1960, los investigadores han construido una serie de lenguajes y entornos con la intención de hacer que la programación sea accesible para un mayor número de personas. Las definiciones de lenguajes visuales han adoptado muchos enfoques, algunos han adoptado un enfoque matemático formal para definir un lenguaje visual (Chang et al., 1986), (Halbert, 1984). Se pueden resumir atributos positivos de la programación visual como lo son una mejor experiencia de usuario (Booth y Stumpf, 2013), una entrada más fácil a los paradigmas de programación (Powers et al., 2006) y además son útiles para prevenir errores sintácticos típicos, debido a la idea de que la sintaxis de las declaraciones ya está implementada en las formas visuales de las declaraciones (Kelleher y Pausch, 2005). ...
El taller recopila elementos de las dinámicas y mecánicas de la gamificación con el objetivo de validar una es-
trategia de evaluación virtual grupal. Tienen como objetivo validar las múltiples estrategias que se presentan en
la formación virtual acompañadas de las virtudes de la gamificación como elemento motivador del aprendizaje.
... Imitation Learning is a promising area of active research. Early research in 'programming by example' began in Software Development [9] before attracting the interest of Robotics and Artificial Intelligence (AI) researchers, who began using the terms 'Learning from Demonstration' and 'Imitation Learning' to describe their line of work. Over the last 30 years, Imitation Learning has advanced significantly and been used to solve difficult tasks ranging from Autonomous Driving [12] to playing Atari games [5]. ...
Imitation Learning is a promising area of active research. Over the last 30 years, Imitation Learning has advanced significantly and been used to solve difficult tasks ranging from Autonomous Driving to playing Atari games. In the course of this development, different methods for performing Imitation Learning have fallen into and out of favor. In this paper, I explore the development of these different methods and attempt to examine how the field has progressed. I focus my analysis on surveying 4 landmark papers that sequentially build upon each other to develop increasingly impressive Imitation Learning methods.
... These demonstrations can then be used to learn strategies that solve the desired problem. A famous class of approaches offering such possibilities is Learning from Demonstration (LfD) [4,7,24,35], which summarizes methods for teaching new behaviors and tasks by demonstration (a) Autonomous Driving [56] (b) Manipulation tasks [3] (c) Activity forecasting [48] 3.1a), robots that solve manipulation tasks ( Fig. 3.1b), and prediction systems that forecast human behavior ( Fig. 3.1c). ...
Da sich die Fähigkeiten autonomer Systemen stetig verbessern, können sie in zunehmend komplexeren Umgebungen immer vielseitigere Aufgaben lösen. Häufig ist es dabei nötig das autonome System an die spezifische Aufgabe oder die jeweilige Umwelt anzupassen, was typischerweise eine umfangreiche Forschung und Entwicklung voraussetzt. Um die Akzeptanz solcher Systeme zu erhöhen, ist es erforderlich deren Einsatz für verschiedene Aufgaben schnell und einfach anhand von Anpassungen der Verhaltensweisen und Ziele durch Nicht-Experten zu ermöglichen. Eine intuitive Art und Weise Aufgaben zu beschreiben ist das Bereitstellen von Demonstrationen erwünschten Verhaltens. Diese Demonstrationen können verwendet werden, um eine Repräsentation der Motivation und des Ziels des Experten zu lernen.
Das Lernen aus Demonstrationen beschreibt eine Klasse von Ansätzen, anhand derer neue Verhaltensweisen trainiert werden können, indem Funktionen und Aufgaben vorgeführt werden, anstatt diese zu programmieren. Zwei Teilbereiche des Lernens aus Demonstrationen sind das Klonen von Verhalten und inverses bestärkendes Lernen. Ansätze aus dem Bereich des Klonens von Verhalten schätzen die Strategie des Experten aus dessen Demonstrationen und lernen somit diesen zu imitieren. Allerdings sind die erlernten Strategien nur geeignet, wenn sich die Umwelt, ihre Dynamik sowie die Aufgabe nicht ändern. Ein populärer Ansatz, der generalisierbarere Repräsentationen lernt, ist das inverse bestärkende Lernen beziehungsweise Inverse Reinforcement Learning (IRL). Dabei wird die Belohnungsfunktion eines Markow-Entscheidungsprozesses aus Demonstrationen eines Experten geschätzt, wobei die Belohnungsfunktion als Motivation oder Ziel interpretiert werden kann. Es existiert eine Vielzahl an Ansätzen des inversen bestärkenden Lernens, die das Problem unter unterschiedlichen Annahmen lösen. Die meisten dieser Ansätze nehmen an, dass ein akkurates Dynamikmodell vorhanden ist, dass das Modell aus Expertendemonstrationen geschätzt werden kann, dass zusätzliche Demonstrationen suboptimalen Verhaltens abgefragt werden können oder dass Heuristiken verwendet werden, um ein nicht vorhandenes Transitionsmodel zu kompensieren. Allerdings werden viele dieser Annahmen häufig verletzt, weil das Dynamikmodel einer Umwelt sehr komplex sein kann, weil akkurate Modelle häufig nicht vorhanden sind, weil zusätzliche Demonstrationen zu teuer sein können und weil Heuristiken die Schätzung der Belohnungsfunktion verzerren können.
Um Probleme des inversen bestärkenden Lernens unter unbekannten Dynamikmodellen zu lösen, stellen wir einen Ansatz vor, der simultan die Belohnungsfunktion und das Dynamikmodell aus Expertendemonstrationen schätzt. Dies ist möglich, da sowohl die Belohnungsfunktion als auch das Transitionsmodell die Strategie des Experten beeinflussen und daher beide aus Expertendemonstrationen inferiert werden können. Demzufolge enthalten nicht nur die beobachteten Transitionen sondern auch die beobachteten Aktionen Informationen über das Transitionsmodell der Umwelt. Mit dieser Arbeit wird eine neue Problemklasse einer simultanen Schätzung der Belohnungsfunktion und der Dynamik eingeführt. Zudem werden mehrere Lösungen des Problems hergeleitet, die unterschiedliche Annahmen an die Generierung der Strategie des Experten stellen. Die vorgestellten Ansätze werden anhand eines Anschauungsbeispiels, der Navigation auf Basis eines Satellitenbildes, sowie der Navigation eines simulierten Roboters in einem Gangszenario mit Menschen evaluiert. Hierbei zeigen die Ergebnisse, dass das Miteinbeziehen der Schätzung des Transitionsmodells in das inverse bestärkende Lernen zu exakteren Modellen der Dynamik und der Belohnungsfunktion führen.
... One major class of PBD systems aims to synthesize task automation scripts so that users do not need to manually write scripting code. These span a variety of application domains including text manipulation [28,49,61,64,74,75], file and directory management [39,65,66], image editing [46,47,53,60], webpage task automation [51,52,58,73], web data scraping [30,50,55], and data wrangling [38,40,43]. Although their domains vary widely, these systems all generate scripts or processed data as outputs. ...
Programmers, researchers, system administrators, and data scientists often build complex workflows based on command-line applications. To give these power users the well-known benefits of GUIs, we created Bespoke, a system that synthesizes custom GUIs by observing user demonstrations of command-line apps. Bespoke unifies the two main forms of desktop human-computer interaction (command-line and GUI) via a hybrid approach that combines the flexibility and composability of the command line with the usability and discoverability of GUIs. To assess the versatility of Bespoke, we ran an open-ended study where participants used it to create their own GUIs in domains that personally motivated them. They made a diverse set of GUIs for use cases such as cloud computing management, machine learning prototyping, lecture video transcription, integrated circuit design, remote code deployment, and gaming server management. Participants reported that the benefit of these bespoke GUIs was that they exposed only the most relevant subset of options required for their specific needs. In contrast, vendor-made GUIs usually include far more panes, menus, and settings since they must accommodate a wider range of use cases.
... Example-based approaches have been frequently used in computer science for assisted learning of difficult concepts [39]. Query by example (QBE) [76] and programming by example (PBE) [27] were proposed to facilitate interaction between computer and users. QBE used examples in the form of query results to guide the process of querying a database system. ...
We present a controlled experiment for the empirical evaluation of example-driven modeling (EDM), an approach that systematically uses examples for model comprehension and domain knowledge transfer. We conducted the experiment with 26 graduate (Masters and Ph.D. level) and undergraduate (Bachelor level) students from electrical and computer engineering, computer science, and software engineering programs at the University of Waterloo. The experiment involves a domain model, with UML class diagrams representing the domain abstractions and UML object diagrams representing examples of using these abstractions. The goal is to provide empirical evidence of the effects of suitable examples on model comprehension, compared to having model abstractions only, by having the participants perform model comprehension tasks. Our results show that EDM is superior to having model abstractions only, with an improvement of 39% for diagram completeness, 33% for questions completeness, 71% for efficiency, and a reduction in the number of mistakes by 80%. We provide qualitative results showing that participants receiving model abstractions augmented with examples experienced lower perceived difficulty in performing the comprehension tasks, higher perceived confidence in their tasks’ solutions, and asked 90% fewer clarifying domain questions. We also present participants’ feedback regarding the usefulness of the provided examples, their number and types, as well as the use of partial examples. We present a taxonomy of the different types of examples, explain their significance, and propose guidelines for manual and automatic creation of useful examples.
