learning project final report

1. Statement of Problem

Traditional classroom lectures, which are typically characterized as "formal learning" and "learning about", are less exciting and interesting as a method to teach topics to children, mainly because they are driven by children's extrinsic motivation[4]. As shown in the video of teaching physics that was played in this class, a live experiment increases children's intrinsic motivation and thus helps them understand the facts and theories behind them. However, not all physical phenomena can be demonstrated in school. Even if we limit the scope to the very elementary physics, for example, it would not be easy to show that the gravity between two objects is propotional to the mass of each object in a classroom.

At first we aimed at solving this problem, especially in the area of physics teaching, by creating an interactive computer simulation using AgentSheets. However, as we conducted our independent research and acquired more in-depth knowledge on motivation factors in learning, we came to think that we should broaden our scope from focusing on physics teaching with AgentSheets to the more general issues of analyzing AgentSheets as a tool to help children learn. This is mainly motivated by the notion that to 'reseed' the findings from independent research into our project should leverage the synergy of doing both the independent research and project in parallel. In addition to this, when we learned about meta-design we noticed that AgentSheets was created based on that concept, giving an opportunity for end-users to design their own applications. If we apply this fact to the field of learning, it opens up a new possibility for school teachers to create an application for teaching, instead of using an existing one.

In this project, we will try to analyze educational use of AgentSheets and evaluate it from four perspectives, that of creator (Alex Repenning), as a software design environment, that of middle-users (school teachers) and that of end-users (school children). Since we have already gained insights on the relationship between learning and motivation from our independent research, our focus is how AgentSheets can increase children's motivation to learn more.

2. Rationale

Nowadays games (or interactive applications) for educational purpose are not uncommon. Phisics Education Technology project run by Department of Physics, University of Colorado has produced many "fun and interactive" simulation applications designed to teach the principles of physics [5]. These applications encourage children to learn the topic by visualizing invisible physical phenomena and letting them interact with the simulation, thus helping them acquire deep understanding of concepts in physics. However, they are static applicaions which neither teachers nor children can modify. It will be almost impossible for a teacher to tailor one of the applications according to the level of understanding of his/her students, unless he/she has advanced skills of programming.

AgentSheets tackles this problem by offering end-user design capability [3], enabling teachers to create their own simulations. Since teachers should know where the difficulty of understanding concepts exist and how to motivate students to learn more, they can be the best designers of such educational applications. AgentSheets should also help teachers visualize theories and concepts difficult to demonstrate in classroom, such as the behavior of electrons. It should be explored further how the synergy of teachers' knowledge and ability to design applications by themselves will enrich children's learning.

3. Evaluation of AgentSheets

3.1. Creator's Perspective

As one major portion of our project, we interviewed the creator of AgentSheets, Professor Alex Reppening. He answered several questions for us ranging from the actual design and implementation of AgentSheets to the types of users it has. We also asked him about some of his philosophies on learning in general so that we could tie in his viewpoints with our research.

AgentSheets is a simulation authoring tool for end-users. Its main focus is on providing the tools and facilities to create simulations or educational games. Alex created this development tool because he felt end-users should be much more empowered than they traditionally are. The exaggerated mission statement he had was to create something "powerful enough to create SimCity in ten minutes." While this is an extremely lofty goal, they really wanted to create something that had a very low learning curve, but was still very powerful and could take a long time to master.

The idea for the actual implementation of AgentSheets came from raw spreadsheets. They wanted to be able to add more than just numbers into a spreadsheet cell, things like buttons or interactive agents. Thus AgentSheets was born.

The target users of AgentSheets are those people that wish to create a simple game, or any kind of scientific simulation but don't want to be bothered by the high learning curve of programming languages. In fact Alex said that AgentSheets was designed to pass the "Pac-Man Test" which means that a simple Pac-Man game can be created with AgentSheets without too much effort. In real applications, AgentSheets is used by everyone from scientists, to the Army, to game creators of all ages.

An interesting finding that they have made in using AgentSheets with different groups of people, in particular adults and children, is that children pick it up more easily. They seem to not be as afraid of making mistakes, so they tend to tinker with AgentSheets as opposed to following a formulaic method of creating something. In some computer clubs the children tend to teach themselves how to use the application, and will share their acquired knowledge with their friends and often create very complex pieces of software without completely understanding the underlying archistecture.

AgentSheets has proven itself to be very successful as it has generally gotten very positive feedback. They only tend to get negative feedback from experienced programmers who expect too much out of the tool. When we asked Alex about the future plans of AgentSheets, he showed us a new tool called AgentCubes that takes the AgentSheets concept and brings it into 3D.

Alex's general views on learning seemed to align with the independent research that we have previously done. He believes that there should be a good balance between formal and informal learning on any given topic. It tends to be the most successful. In his classes he allows the students to examine other students' source material in the hopes that they will learn something informally. He also stressed the importance of motivation in learning something, especially a foreign language. Without some kind of need to learn something, or if the learner doesn't understand why something must be learned, then they will rarely learn it in the formal setting. His most insightful comments came when we asked him the difference between adult and child learning. He feels that it's all about their different attitudes. Children are explorers and inventors, and while they may be more prone to make mistakes, they often learn things with a lot less scaffolding than is often required of adults. Alex feels that the school system kind of hinders this explorative attitude to remain in a person as they grow up and go through school, and this has a negative impact on the rest of their learning throughout life.

3.2. AgentSheets as an Application Design Environment

As revealed in the previous section, AgentSheets was developed as an enhancement of spread sheet concept. What a developper need to do to design an application on AgentSheets is to define the behavior of an Agent in the graphical programming environmant. Similar to spread sheet applications, an Agent resides in a cell on a worksheet and it interacts with other Agents. Based on its surrounding condition, it can change its state, move to a neighboring cell, or even disappear. This paradigm is quite suitable for creating simulation applications, either physical or social, since there is a one-to-one relationship between an Agent and an entity in the simulation. What an application designer should do is just to implement the behavior of an entity as an Agent in AgentSheets.

Form for defining Behavior of an Agent

This model is also quite similar to that of object-oriented programming languages such as Java and Smalltalk. An Agent is analogous to an object in these programming languages in that they both have their states (member variables) and behaviors (methods). Therefore, if an experienced programmer is familar with OO languages, it should be quite easy to understand the concept of AgentSheets and to create an application on it.

One possible drawback of AgentSheets for experienced programmers may be its heavy reliance on entirely graphical programming environment. Unlike macros in Microsoft Office or Automator[6] in Mac OS X, AgentSheets does not have associated programming languages such as Visual Basic for Applications[7] or AppleScript[8]. This makes it clumsy to create complex behaviors, since every single condition/aciotn in a behavior has to be dragged and dropped to define them. The lack of text programming environment also implies that a user cannot learn how to program with AgentSheets, thus making it a "tool for living". In case of macros in MS Office, it is possible to convert a macro into codes and a user can learn how to write a code to achieve a desired effect. Introducing a kind of scripting language into AgentSheets would give a possiblity to add an element of "tools for learning". This also matches the AgentSheets' motto of "High ceiling".

3.3. Middle-User's Perspective

After thoroughly interviewing a middle school science teacher, Alice Dale-Thomason, it became quite apparent that AgentSheets has much to offer teachers and students. Alice preferred to keep her students engaged with live examples and experiments. However, there are many concepts in that experiments which are not readily available. For example the interaction between protons and electrons cannot be visualized. Agent sheets provide teachers the ability to easily create a simulation demonstrating these interactions. It also gives children the fun and exciting visual effects to keep them engaged in the subject. Alice noted that concepts that are not easily visualized are often the most difficult for her students to understand. We feel the use of AgentSheets will gap the bridge between visual and concept learning.

There are many features that make AgentSheets a great learning tool for students. Animations and sound effects have the ability to awe the children and learning becomes fun. Alice agreed that the explosions and sound effects will be fun and exciting for children, it makes the application for learning more of a game than a learning tool. She said the children would really like the car crash and explosion in the bridge builder example we showed her. Alice is very interested in using AgentSheets to aide subjects that are physical experiments and examples are not readily available.

3.4. End-User's Perspective

To evaluate the Ender-User's perspective on AgentSheets, we decided to employ the aid of two High School students. One student being a Freshman having never taken any type of Engineering or Civil Engineering course. The basic classes for this particular student included introductory physics and math. The other student was on the opposite side of the spectrum, being a Senior who had also taken a few engineering courses and civil engineering courses offered through the High School. Both users were instructed to use the "Bridge Builder" AgentSheets application with the minimalist amount of instruction and explanation of the application. The users were given information about using the tools and the representation of items in the gallery, but were only told to meet the goal of the application. To construct a bridge that could support the application's loads with the fewest number of stone blocks. After using the application and attempting to meet the criterion of the "Bridge Builder," the users responded to a few questions about the application. These follow-up questions corresponded to similar questions asked of the users prior to their use of the AgentSheets application. Here are some questions and repsonses to gauge the users' knowledge prior to using AgentSheets:

What do you know about bridges?

Freshman: "It has a suspension to hold up heavy objects."

Senior: "A bridge must also be able to support it's own weight, live loads and dead loads."

Observations made about both users revealed a general understanding from both about bridges. Each one was familiar with what a bridge is and both recognized the structures of existing bridges with references to suspension cables and structured support. However, the Freshman tester was less restrictive in design and creation. Going outside the bounds, he tended to do abstract things like filling in the valley with dirt and proclaiming that the objective of using the least number of blocks was still met. He also resorted to trying to build less conventional structures, like using triangular shapes or any other pattern and was less hampered by the concept of support and gravity. On the other hand, the Senior tester conformed to more well known and already in use shapes and designs. He attempted to build an "Arch" shape with the blocks and felt constricted by the application because he was unable to build a suspension bridge with the limited items in the gallery. He refered to more specific terms like "Live Loads" and "Foundation" in his description of bridges. In the end however, neither one claimed to have learned anything new that they didn't already know before hand. The Freshman tester went further to say that it would've been helpful to have a teacher guide the process and then perhaps he could've learned more.

From using the application ourselves, we came to see many similar occurences throughout all the many different AgentSheets applications. They all share the same User Interface. There is a main canvas where pre-programmed behavior occurs. On top of that exists a gallery bar containing the different objects and items that can be inserted into the canvas. To the left is a toolbar that contains all of the same tools, a pointer tool, a pencil/draw tool, an erase tool, a selection tool and other various paint-like tools. At the bottom is a user feedback text box that informs the user of errors and events that cannot be interpreted by the application. Such a message pops up when using the "Whack" tool on items in the applicaiton for which that event has not or was not programmed. Using the applications all present a dilema in that users do not readily know what each item in the gallery does or is capable of doing. Furthermore, using the tools is very non-intuitive as they behave like Paint tools, but the canvas is not perceived in the same way. Users are not expecting to "Draw" an item onto the canvas, but rather they expect to place and drag or drop items as they go. Furthermore, as an item appears on the canvas, users are tempted to click on it and select it while this function is not supported without first switching tools to the pointer. The process to draw and erase an item requires selecting the draw tool and then switching to the erase tool. This makes it obtrusive to the design process as users are constantly switching back and forth between tools. Although the goal of AgentSheets is to make it easier for non-programmers to program an application, it does not make it easier for non-programmers to use the application.

3.5. Conclusion

After fully evaluating the effectiveness and potential of AgentSheets as a programming tool and a teaching tool, we have come to a few conclusions. For programmers looking to create applications with AgentSheets, there are several benefits. Creating an AgentSheets application is simple and can easily be done by anyone looking to provide a simple simulation or model for teaching purposes. There is overall, a shallow learning curve with using AgentSheets that can potentially yield non-trivial simulations and applications. AgentSheets is and can be a very powerful teaching tool to help aid student learning in a classroom environment, as well as being applied to many different contexts and uses. The complexity of applications built using AgentSheets can be quite deep depending on the experience of the programmer. While as an application builder it is very useful, there are additional facets to the experience as an end user of the application built with AgentSheets. The outcome of using AgentSheets on a user�s perspective can be very positive. The applications can be very motivating by being fun and interesting to user, especially young children. However, there are some negatives to using AgentSheets as a tool to teach. Including the ability to add code would be beneficial for programmers with some experience to provide additional functionality not currently found in AgenSheets. Furthermore, the common GUI found for all the applications is not always intuitive. Functionality is also limited by how the programmer builds and sets functionality for the application. In the future, many of these shortcomings will be addressed with the introduction of AgentCubes, a 3D programming tool with the same concept of simple programming and high functionality. However the challenge will be to provide a User Interface that is easy to program for.

4. References

[1] Don Norman, Learning from the Success of Computer Games http://www.jnd.org/dn.mss/learning_from_t.html
[2] AgentSheets http://www.agentsheets.com/
[3] Alexander Reppening, "AgentSheets: an Interactive Simulation Environment with End User Programmable Agents"
http://www.cs.colorado.edu/~ralex/papers/PDF/Interaction2000.pdf
[4] Learning Final Report learning final report
[5] Physics Education Technology
http://www.colorado.edu/physics/phet/web-pages/index.html
[6] Automator
http://www.apple.com/macosx/features/automator/
[7] Visual Basic for Applications
http://msdn.microsoft.com/isv/technology/vba/default.aspx
[8] AppleScript
http://www.apple.com/macosx/features/applescript/
[9] Interview with Alex
[10] Interview with Alice
[11] AgentSheets Questionaire/Survey results

Learning Group ― Final Project Report