![[spacer]](/schemes/blue/line.gif){kind=small}
|
Collaboration in Software Development Process Rizwan Ansary, Randell Rivandeneira, Andrew Skalet, Nilo Tsung Independent Research Final Report I. INTRODUCTION Software development has been a difficult human endeavor since its first days. Fred Brooks documented many of the problems we still face today relatively early on in his excellent book, The Mythical Man-Month. However, the endeavor has become even more difficult over time as well. While early projects were often end-to-end contracts intended to solve a specific and relatively well understood problem, modern projects often face uncertain and changing requirements, and long term maintenance and modification through several versions. We believe that many of the difficulties of the software development endeavor are, in essence, issues of Design, Learning, and Collaboration, of the people within the project. Traditional methodologies of software development have shown themselves to be ineffective in solving these problems in many environments. Even worse, using no methodology at all has proven its trouble over and over. Furthermore, in the future Design, Learning, and Collaboration issues will become more difficult as we have users much less experienced in design doing development, and needing to collaborate across long distances. We believe that many of modern development recipes that have emerged to combat these problems are really addressing Design, Learning, and Collaboration issues. We have chosen to focus on “Extreme Programming” in our research, because it is very different from traditional practices, but is also proven in some environments. Extreme programming (XP) attacks a number of Design, Learning, and Collaboration problems that exist in many development environments. XP improves design in a number of ways. XP requires very small releases, so teams only have to plan for the most critical business requirements at the time of development. This greatly simplifies the design process, since it is much easier to determine a few highest priority items for the next month than to determine many moderate priority items for the next year. XP also intends to create the simplest systems that still solve the problem, to simplify designs as much as possible. Furthermore, XP acknowledges that programmers need to rework troublesome code from time to time, and calls that “Refactoring.” Finally, XP requires tests for all code, which improves Design through direct feedback. Extreme programming also improves collaboration in meaningful ways. Most obviously, XP requires that all production code be written using one programmer and one observer sitting at the same desk. This directly implements collaboration to solve problems more effectively than a single person can, and improves the learning process through collaboration. A metaphor, or “simple shared story” is created in an XP project to improve collaboration by starting everyone on the same page; with the same idea of the project. XP also enhances collaboration in software development by asserting that all code can be worked on by all programmers; this removes authority and personal barriers to collaboration. XP improves collaboration with non-programmers as well. XP requires that there be a “real life customer” that acts as part of the development team. This introduces collaboration between engineers and a customer directly, which makes the customer’s needs far more clear to the engineers. The practice of extreme programming is no panacea, and is more effective for some environments than others. However, it illustrates clearly that improving software development processes has quite a lot to do with Design, Learning, and Collaboration. In order to understand the extreme programming approaches better in a real world context, and compared with other modern development techniques, we interviewed senior developers with experience with modern and extreme programming practices. We lack the space to describe their reactions in detail, but their verbatim responses are attached in appendix A. II. EXTREME PROGRAMMING: SRATEGIES Whether it is transitioning an existing project or starting a new one, implementing extreme programming practices require planning and a concerted effort. In order to avoid deadline slips and accommodate changing requirements a certain amount of flexibility is required. This is exactly what XP aims to achieve. The productivity can be maximized during each phase of the project if a set of certain guidelines are followed.
III. PAIR PROGRAMMING Pair Programming occurs when two programmers work side-by-side at one computer collaborating on the same design, algorithm, code or test. This practice helps to achieve higher quality products that are produced faster. Yet most that have not tried and tested pair programming reject the idea immediately as an overtly redundant, wasteful use of programming resources. One important result is that those who are involved in Pair Programming have an enjoyable experience in problem-solving and are more confident in their solutions. I have several papers that ranger from the anecdotal to the experiment set that proof the previous statement. Pair Programming has been used for a long time, yet its popularity came out when Extreme Programming (XP) software development methodology started evolving in 1996 by Kent Beck, Ward Cunningham and Ron Jeffries. They credit that much of XP success is due to the heavy use of Pair Programming by all their programmers, experts and novice alike. How it works? One partner is the "driver" and has control of the pencil/mouse/keyboard and is writing the design or code. The other person continuously and actively observes the work of the driver: watching for defects, thinking of alternatives, looking up resources, and considering strategic implications of the work at hand. The roles of driver and observer are deliberately switched between the pair periodically. Both are equal, active participants in the process at all times and wholly share the ownership of the work products whether they be a morning’s effort or an entire project.
Finally, there are some good practices for people who want to experience pair programming. Please refer to the power point presentation in pair programming for the full description of good practices. IV. A CONSTRUCTION MANAGEMENT PERSPECTIVE Although product types in software industry and construction industry are very different, both these two industries are project-driven. A general distinction between a project and a program is that the former usually has a defined life with a clear beginning and end, while a program has a continuing existence until it is brought to an end. The principal features of a project are
The basic tasks of management are planning, measurement, and control. However, there is a great difference between the management of projects (e.g. construction and software industries) and the management of production (e.g. manufacturing industry). Project managers tend to look forward and try to foresee uncertainties, while production managers almost always look backward and minimize variability. Many writers have addressed questions such as "why so many projects fail?" and "how can we measure the progress of a project?" Unfortunately, the answers for these two questions are too many and too diverse. Based on my limited knowledge and experience in these two industries, I believe if we can achieve high degree of customer satisfaction, business, in both construction and software industries, will come to us more than we can handle. It is especially true when customers require projects be finished in a reduced time but provide us with vague and changing requirements. Construction industry has been attacking this problem by adopting a management process called project partnering since early 1990s. Partnered projects start with an off-site one- or two-day partnering meeting prior to the preconstruction events. The partnering process is as follows:
Project partnering as described above shares many similarities with extreme programming discussed by other people in this team. Partnering in construction industry has been widely used in many projects for more than 14 years in the US, and has many successful stories. Most importantly, it has established an effective and systematic way for communication, training, meeting, trust building, etc. Software industry may learn and adopt the practice of project partnering from the construction industry in order to meet future challenges. Flexible organization is one of the key elements in order to have a successful project. SimVison is a computational simulation tool for virtual team design. It can help the project manager to analyze whether a team can finish a project within a reduced time and predict effects on project when the organization of project team is changed. Due to the limitation of time, this tool has not been studied too much. However, it seems an appropriate tool for analyzing a flexible organization for a fast-tract project. More information about the tool and the underlying model can be found in the web site of Vite Corporation. V. CONCLUSION Software development lifecycles are not without unknowns. Producing quality software is a challenging task and at times, the failures can be frustrating. Extreme programming addresses the specific needs of small teams that have to deal with vague and oft changing requirements. A project idea that is new to the industry involves high risk and hence, is also a candidate for extreme design and development. However, XP is not a universal answer to all software engineering problems. It too can be frustrating, especially when used in the wrong place. Transitioning to extreme programming is a tricky business and should be handled practice by practice. Programmers schooled in the traditional model of software engineering are slow in jettisoning long standing notions and unwritten rules that extreme programming breaks on a regular basis. Once at ease with XP practices, programmers have usually enjoyed their jobs more than they used to before. VI. ACKNOWLEDGEMENTS We would like to thank the following people whose inputs have been valuable in the production of this work.
VII. REFERENCES 1. Beedle, M., and Schwaber, K. Agile Software development with Scrum. Prentice-Hall, 2001 2. Beck, K. Extreme Programming Explained. Addison-Wesley, 2000 3. Beck, K., and Cooper, A. Extreme Programming vs Interaction Design. FTPOnline 2002 4. Fischer, G. et. al. Putting Owners of Problems in Charge, ITR proposal, 2004 5. McConnell, S. Rapid Development. Microsoft press, 1996 6. Baheti, P., Gehringer, E., and Stotts, D. Exploring the efficacy of distributed pair programming. 7. Williams, L., and Kessler, R. All I Really Need to Know about Pair Programming I Learned In Kindergarten. 8. Williams, L., Kessler, R., Cunningham W., Jeffries, R. Strengthening the Case for Pair-Programming. 9. Succi, G., and Marchesi, M. Extreme Programming Examined. Addison-Wesley 2001 10. Martin, R. Agile Software Development: Principles, Patterns and Practices. Prentice Hall 2002 11. Erdogmus, H. Comparative evaluation of software development strategies based on Net Present Value. 12. Kunz, J. C., Levitt, R. E., and Jin, Y. (1998), The Virtual Design Team: A Computational Simulation Model of Project Organizations, Communications of the Association for Computing Machinery (CACM), 41(11), November, 1998, pp. 84-91. 13. Pressman, R. S. (2001), Software Engineering: A Practitioner’s Approach, McGraw Hill Co., Boston. 14. Extreme Programming Website, http://www.extremeprogramming.org 15. Vite Corporation, http://www.epm.cc/. Appendix A 1. What do you think are the most important elements of planning a development cycle? Scott: First and foremost is to understand the problem you are trying to solve. Developing software without a clear understanding of what the final product should be is setting yourself up for longer schedules. Pete: Feature Clarity. Knowing what has to be done and why. Planning features, especially in any sort of iterative process, need good ordering too. This included what features impact other features, relative amount of risk in the feature (chance of time overruns, instability, feature creep, etc). Come up with a plan that minimizes the risks and is logical for refactoring when there are feature collisions. A good general idea of how features are going to be implemented and what they impact is necessary for all of this. 2. Do you think an iterative development approach is always a good idea? Scott: In terms of iterative approach for XP, I think it's a very effective way to develop. It gives developers the chance to have a sense of completion at the end of an iteration. Plus, if done correctly, your "users" can expect to have a functional product at the end of every iteration. This provides for rapid feedback. Pete: I've been leaning towards away from a strict iterative development approach lately, but I think it really depends on the application. When there is a serious need for stability and features are often quite small and well defined, I think the iterative cycle is still quite good. For applications like Anark, we have a significant number of large tasks to implement which impact large portions of code, and we know that there is going to be a lot of feature creep. Our features are much more difficult to predict, both from the users standpoint as well as the full capabilities of them. So I like to do the heavy construction at the beginning of an iteration, put the framework of the major features in at the start, but leave the code in an unstable state. Then bring areas up to speed in the order of importance. This allows us to handle user feedback on the critical sections early on, but have all core work finished up well before the release. The feature is not complete as in the iterative approach, but is instead stable on the back-end and somewhat done on the front-end. An iterative approach would be to say that you make the back-end solid, then implement the front-end, but that never works out properly. The danger is that you can get close to your release time and have many features in a half-state and be completely screwed. So if this is done, then conservative dates need to be set for the critical back-end work to be done, and ways to disable features easily from the front-end should be implemented. 3. Do you think it's important to create a "shared vision" or metaphor for a project, when it begins? If so, do you think that it is necessary to have a single person that is the "vision keeper" (probably not a manager) for the project, or should a consensus approach be taken instead, with no central point of contact? Scott: Having a shared vision is nice because I feel the team can always have that vision in mind, which will enable more creativity for possible new features that may have not been thought of before. For example, if I know that I'm creating a data entry application for a bank I might think of things a bank might need to produce a better workflow. If it was just a general purpose data entry application, I might not be thinking of everything at the time of design. I think it's important for somebody to oversee that features going into the product jive with the overall vision. Whether or not that's a manager, I'm not sure. Pete: I'm all about the "shared vision" with one person setting the direction. Too many visions make a very fractured program. Of course everyone's visions need to be considered, but there should be one person driving the product. It keeps the product going in the same direction. This is a real-world thing, in certain ideal situations you may have a core group who consistently share the same vision, but that doesn't happen too often. Disagreement about features will always happen, and that's great. Everyone working on the product should be considered very experienced users so their opinions really matter. But when it comes down to it, not everyone can take the time to interview all the customers, do market research, etc. The one person driving the product is responsible for the direction, and they are the most critical person in the company. 4. Should designs be kept absolutely as simple as possible until it is necessary to add additional functionality? Is it a good idea to leave some room in the design for anticipated enhancements? Should programmers have a little room for creativity, as well? Scott: Over engineering code can happen very easily when your design is trying to take into account every type of use case scenario. Not that I don't think some hard thought should be put into a design, but a balance has to occur. You're more than likely not going to think of everything at design time, so just make your design something that is flexible and "easy" to change if need be. Good object oriented techniques can be used to help enable this. Software is "soft" and is easily mandible if need be. Pete: Keep it simple. Always. But I believe that the simplest solution is most often the most flexible one too. A well designed system should always be flexible. It doesn't mean that you have to implement the flexibility immediately, but just have the proper abstractions and such that you can always move stuff around. This is the most important part of designing large software as far as I'm concerned. Features will ALWAYS change, always. And there is no better feeling than being in a system that you know that any change you need to make is as easy as modifying a few points of code and having everything change. I like to put in a bit of flexibilty up front in features that I may or may not use, just to ensure that design constraints are maintained in the future, and to keep up the general principle that flexibility is super important. Pete: Programmers should definitely have some creativity. It's better for moral, for the idea that the code they write is theirs, and for general pride in their work. Just make sure that it does not get out of hand. Also this really impacts refactoring, there needs to be flex time in programming, not all work should be micro-managed. Allow some flexibility and trust that your engineers are good enough to make the proper decisions as to what needs to be done to make an excellent system. 5. Extreme programming says that all code should have unit tests associated with it, and that all tests must pass before any new components can be developed. Do you see any downsides to this? Scott: Unit tests are great in theory. The only downside I see is the time it takes to make them. In theory it saves you time in the end by "catching" bugs, but I honestly do not have much experience with them due to the time it takes to make them. It's something everyone at the company has to embrace, including the higher ups for scheduling purposes. Pete: Yeah, there's a lot of code that if it doesn't work is completely obvious as to what is wrong when it breaks. And maintenance can be a nightmare keeping up the unit tests. I really like unit tests for small, critical systems that cause errors that can be really tough to track down. Then acceptance tests for larger sections. Good acceptance tests I think are more critical and more efficient from the implementation standpoint. Unit tests are great for tracking down a bug to it's exact spot immediately, but if acceptance tests can locate them in minimal amount of time (2 mins to find a bug) then I think the acceptance tests are acceptable. It completely depends on the application though. Unit tests for every line of code is quite extreme and assumes very ideal circumstances. 6. XP also adopts "Refactoring" as a core value. Isn't this something that has been done since the dawn of programming, when a programmer realizes some section of code sucks, and it needs redesign? If not, how is Refactoring different? Scott: Yes, refactoring is something every good programmer should do. The difference I feel is that the XP process embraces it. If your development process does not embrace refactoring then theoretically your schedule can never take that into account. Pete: Refactoring is nothing new, I guess XP just formalizes it as part of the process and puts more emphasis on it as being necessary. This is good, refactoring is extremely important, so anything that improves it should be encouraged. 7. XP also says that all code should be written with 2 programmers sitting next to each other. Do you think this is always a good idea? If not, should programmers working alone much of the time have an open work environment, or private offices, with separate meeting rooms? Scott: I feel that an open environment is very conducive to getting help from fellow programmers. Therefore, pair programming sort morphs into whatever your environment allows for. I have no experience of having a separate office, but I would guess that pair programming would be effective in this situation. Pete: I hate pair programming. Some people really really like it, others do not. It comes down to personality and how you work. Closed offices with one person per office is extremely good when you have good engineers who write solid code and are working on a system that they know quite well. And I think this system makes the best code too. I think pair programming is for developers who are not quite aware of the entire system they are working in and are not quite as familiar with the requirements as they could be. Not every engineer will be top-notch, so this happens. But no matter what, I don't see the stability and efficiency of two engineers matching that of one solid engineer. 8. XP promotes "collective ownership" as well. Do you think there are cases where people should be experts in certain sections of code, and that others should not work on that code? Perhaps 2 people who specialize in an area could know it, but not the whole team. Scott: Not all programmers on the team have the same knowledge as the others. If there is a piece of code that is math intensive and not everyone has a knack of for complex math, than it does not necessarily make sense for everyone in the team to learn math. Some people are experts at certain things and therefore get tasked with writing those specific pieces of code. However, I feel that if any programmer wants to learn those specialized code sections, the process should give them the opportunity. Pete: Code ownership requires a fine balance. Everyone should be familiar with all parts of the system, not having this will have too much duplicated code and systems will not work together very well at all. People should always move around the system, this takes a lot of boredom out of implementing features and encourages you to do a good job if you know others are going to be in the code as well. But I think that there should be owners of sections who are responsible for the overall design. Just as I think one person should have the vision for a product, one person should have the vision for a section of code. This ensures that the code is always consistent within itself, and also makes a goto person to talk to about new features and how they can best be implemented into the system while maintaining integrity. If you have one person who knows everything going on in a section of code, then they can make much better suggestions for refactoring, when it should happen and how. 9. XP also says that people should always work a 40 hour week, and nobody should ever work overtime 2 weeks in a row. Do you think this is possible, especially prior to a release? Scott: I do think this is possible. As times goes on your methods and processes get refined. In the beginning I feel that every company will go through these 2+ week crunch periods. However, as we learn from our mistakes and adapt the process, time crunches can disappear. Pete: Under an ideal world, yes, this is possible, and many people like it. I don't. My ideal work situations are to go balls out for a month, then have 2 weeks of slow time. When I'm working hard, I don't want 40 hour weeks, I want to be working. And when there's an off time, I want to take long lunches, sleep in and leave early. Many developers like it this way, but many do not. It's personal how you work and a 40 hour week is very arbitrary. 10. XP says that a "real customer" should be part of any development project. Do you think this is realistic/helpful? Scott: This is very true. Ideally you want to have the customer sitting near the programmers, using the product while you develop. That way you get real-time feedback on what is useful, what could be different, bugs found, etc. I don't know how realistic it is for every piece of software. Sometimes your software caters to many different types of users (or a whole industry) and getting many different users to use different parts of the product might not be as easy. However, if it was possible, I think it would help. Pete: My experience with real customers is that they don't know what they want, don't want to take the time to really use a beta/alpha product and when they do, they give very shallow feedback. The person with the overall vision of the product should be the equivalent of your uber-user and should know everything about the customer necessary. This person can talk to users, give them betas for testing, but they will most likely have to guide the users through the process and provide some interpretation of their feedback. 11. Are coding standards ever a problem? Can they go too far? Scott: Coding standards are very useful and should be embraced by the process. They have a tendency to go to far at times and this is a sign that they need to be reworked. However, you're never going to make everyone happy so find a balance and stick with it. Pete: Coding standards are good, just to ensure that all the code is somewhat consistent, and that you know where to look for information and methods/variables are predictable and consistent. It just ensures that the system is consistent as a whole. Coding standards that cover formatting and gets into small details is way too draconian and unnecessary. I hate Anark's coding standards, there's stuff in there that just doesn't make sense and acts as a detriment to the code. Strict coding standards offer little advantage and remove developer's personalization of code (which I think can be a good thing). |
