I don’t want to make a practice of this, but I thought this post was so brilliant that I had to add it to my blog so I could remember where to find it later. Nicholas C. Zakas wrote a piece entitled, “The care and feeding of software engineers (or, why engineers are grumpy).” It hits the nail on the head in many respects. If development and product managers would follow the advice listed, many a software engineer would be much more content. Happy reading! The care and feeding of software engineers.
A few posts ago, I spoke about using a version control system in order to provide you with several advantages while developing. The next piece of software that deserves attention is a Continuous Integration System. As with my version control post, we’ll first go over theory regarding why it is important and how it can be used. We’ll then go over some of the products available and end with a case study on one of my projects where my team reached the holy grail of development automation by combining a version control system with continuous integration.
The difficult part of writing about continuous integration is that there isn’t one logical place to start. Any one comment made about it would also be interrelated with 2 or 3 other ideas that each deserve their own piece of explanation. I’ll do my best.
At it’s simplest, a continuous integration tool is something that allows a team of developers to follow best practices regarding the integration and deployment of their code. When I say “allows,” I mean there are many decisions to be made and the initial setup is a very manual process. You have to decide on what “best practices” are for your team based on the language you develop in, your server setup and plans for the future. A continuous integration tool depends on having a couple other pieces of infrastructure already in place to be useful. One is a version control system, which we’ve discussed, and the other is a build tool, such as Ant, Maven, or MSBuild. I also want to mention that continuous integration has slightly less meaning in an environment where you are not compiling source code, such as PHP or Coldfusion, but can still be used to great effect there.
The setup effort can reward both the development team and the management team with things that they want. I’ll start with the developer’s point of view.
- Automated Source Code Compilation. One of the biggest sources of frustration for a team working on a compiled product is a developer committing code that breaks the build. Continuous integration can be set up to watch the version control system for changes. Whenever a developer commits to the mainline, continuous integration will kick off a build. Continuous integration will report on the status of every build, and can be set up to send out emails with those reports. Build problems can be rectified immediately.
- Automated Unit Testing. If the development team is using test driven development, the continuous integration system can be set up to run those unit tests automatically. Developers can know if someone on their team modified code that caused test to fail. The developer and the commit can be identified and the situation can be rectified before it becomes a large problem.
- Automated Deployment. In addition to compilation, the continuous integration system can be set up to push compiled code and other necessary assets out to a server. This is where the rubber can really hit the road and save you a ton of time. Ideally, you’ll want your automated system to push to a development or integration box first where developers can test that their code works with the code of everyone else on their team. Next, the CI server can push it out to the staging or UAT (User Acceptance Testing) environment. This is where the QA and management team can verify requirement completeness/correctness. Lastly, the CI server can push it out to the production environment. This can all happen in an automated fashion without the developer needing to babysit.
Next, there are some benefits that managers get.
- Happy Developers. Developers get to spend their time managing coding on the problem domain instead of spending their time babysitting deployments. Developers like to solve problems, but generally not organizational problems or problems that have been solved time and time again. In general, developers also don’t want to participate in processes fraught with the possibilities of human error. There’s always a bit of trepidation before a manual production deployment.
- Increased ROI. Automation saves time and money. Many factories across the US have replaced much of their human workforce with an automated workforce. Regarding continuous integration tools, a setup time of 2 or 3 developer days and some minor maintenance can save developer weeks or months over the course of the year. And what manager wouldn’t be happy to get a bonus due to the increased productivity of their staff?
- Error Reduction. If you have a 10 or 12 step production deployment process (which I have participated in before, so it’s not out of the question), there’s a huge possibility of human error on one of those steps. Automating that process gives you consistency and virtually removes all possibility of error from the process. Consistency regarding production pushes makes clients happy, which increase sales, etc.
- Automated reporting. Continuous Integration tools can give managers additional insight into development status, deployment status, and whether something may need their attention. If one particular developer continually breaks the build, you may need to get them additional training, help, support, etc. Conversely, if a particular developer never breaks the build, you can reward him/her. Additionally, these systems can give you the insight you need to report to clients without having to have numerous staff meetings or questions to figure out the status of a particular project.
If you haven’t guessed yet, I am a huge fan of continuous integration systems due to the many benefits they provide at a relatively low overhead and maintenance. In the next article, I’ll be looking into specific tools available and under which situations you may want to use them. I’ll also touch on the correct server and development computer setup to make maximum use of the technology.
The famed “death of flash” blog postings and speeches have been making their way like wildfire around the interwebs. It doesn’t show any sign of stopping soon. I think a big part of this is due to fear on the developer end of things. Nobody wants their technological livelihood to disappear. Another part is the lack of clarity on Adobe’s end. Anytime they publish something regarding the direction of the platform, it has people asking more questions than it answers. Take a look at the Flex Team blog post here for an example.
Because this technology affects me and my preferred development environment personally, I’d like to touch briefly on it as well. However, I’m not going to make any predictions, but rather try to imagine where we’d be if the flash platform were to disappear.
The first point I’d like to make is that I think the quality of web development would go down across the board. I know, it sounds crazy, good old Steve told us that flash==memory leaks/bad programming/etc. But hear me out. When I began as a web developer, I had a natural inclination toward architecture. I’m a big picture guy. When I understand high level architecture, it helps me make the most minute decisions better because I understand how the pieces fit together. So, I did my research and attempted to learn design patterns and MVC.
At the time, however, I was having trouble reconciling MVC with the page refresh and stateless nature of HTML. I wasn’t around anyone that could provide me a good example of MVC in my development environment (Coldfusion, and for the life of me I couldn’t understand Mach-II). I began experimenting with Coldfusion forms, which was really a subset of Flex 1.5 functionality stuffed into a Coldfusion server. I liked this a little better because I no longer had to refresh the page every time I wanted to change data.
The web would still be ugly in the area of applications as well. I think in this regard, Flash and Ajax pushed technology forward, but one would never have gotten as far without the other. One of my beefs with the user experience when I first started development was always having to refresh the page and redraw everything, just to change a little bit of data. Flash and later Flex enabled huge breakthroughs in UI development. We could now have designers and developers working together to create awesome user experiences, in the enterprise and for consumers, with no page refreshes to slow things down.
Web video. Would we even have a Youtube or Hulu without Flash? Flash was the first platform to enable easy development of video players coupled with the server technologies to truly protect your content.
This post will be quite listy as it is simply a description of the roles you need in place to build a successful software product. These are roles, not titles. There are no rules that state you cannot have one person doing multiple roles. If you’re a one man/woman development shop, you’ll find yourself a lot like the new Staples commercial here. I’ll try to list them in the order they would typically appear in the process. Some will have just a small role in one part of the process, while others will span the entire process. Please note that my short blurbs are very high level and will likely not be representative of all the duties of that particular role.
- Stakeholder/Customer/Client – This is the person or entity that is requesting the software. They would be involved in the initial concept idea and provide funding for the project. They would also be involved after the completion of major phases of the project, such as analysis, design, development, and quality assurance, to provide feedback that the software is meeting requirements.
- Project Manager – This role maintains the timeline for the project and ensures that the correct resources are available when they need to be. They ensure that the project meets timeline, budget, and quality standards. The project manager is the most critical person in the project because they will need to be involved in all phases from initial concepting through to deployment or delivery.
- Business Analyst – This role creates requirements for the software. They are responsible for taking information from the stakeholder and translating it into specifications that developers can use to build the system. Ideally, the requirements documentation would be another touch point for the stakeholder to determine whether or not they are going to get what they asked for. After stakeholder sign off, these specifications are used by the quality assurance team to ensure that the delivered product meets requirements.
- UX Designer/Information Architect – This role creates and maintains the wireframes for the product. They take the specifications and translate them into something graphical that the designers can begin to work with. They capture each of the requirements provided by the business analyst and lay the initial plans for the functioning piece of software.
- Designer – This role creates the design of the software product, combining efforts from both the Business Analyst and the UX Designer. The deliverable is graphical files from which the developers can begin their work. After the designer completes their work, this would be another point for the stakeholder to approve continuing to move forward. Additionally, this role may be involved in creating graphical assets from their design work that will be used in development.
- Art Director – This role monitors the work of the designer to ensure that it not only meets modern design standards and is aesthetically pleasing, but also that it conveys the tone or message of the stakeholders properly.
- System Architect – This role is made up of a few parts. The first one is to ensure that the necessary systems and resources to complete the technical part of the project are available. This includes servers, source control, development machines, developers, etc. The second is to estimate the development effort of what has been provided by the prior roles. They need to take into account design, user experience, and requirements. They architect the development and provide a UML diagram to the developers. Lastly, they monitor development to ensure it is meeting expectations.
- Developer – This role takes the UML diagram from the architect as well as the design from the designers and builds the actual product under the direction of the architect. Because development is generally the longest phase of the process, progress meetings should be set up at regular intervals. The developer should be self-testing their code to ensure that it meets business requirements set forth by the business analyst.
- QA Analyst – This role tests the work completed by the developers to ensure it meets the designer, business analyst, and functional specifications. They are the last line of defense before delivery. It is important that the time necessary to complete this step makes its way into the overall project plan.
The one other role I considered, but did not add, is that of a technical lead. Depending on your organization, you may need a person between the project manager and the technical team to ensure both groups are on the same page during the entire process. This role could be filled by the architect, but you may break it out separately as well.
So there it is. Some projects may use everybody in the order listed, but the reality is that the majority of the time, software development is an iterative process. Stakeholders may not know that they don’t want what they said they wanted until the see it in front of them. It’s very important to keep them in the loop.
It’s also important to do your iterations at the point where it’s easiest. This means that you may need to go through your requirements or wireframes a couple of times. It’s easier at that time then during the design phase. It’s easier to change a design than a full-blown application.
Lastly, your project will be most successful if you don’t skip any of these roles or their deliverables. Many organizations have a tendency to skip one or more to save time, and it ends up costing more time and money at the end. Following all the steps gives you the opportunity to make changes when there is the least effort involved. However, you may alter the fidelity of your effort at each of these steps as your project needs dictate.
In a previous article, part 1 of the series, I went through a few of the reasons you might want to consider using source control in your development infrastructure. I’d like to continue on that article with what I promised, some concrete ideas of source control systems you can consider implementing. I hope to highlight some of the major advantages and drawbacks of each system as well as give some context to help you make your decisions.
Here are the major players in the open source world:
- Subversion – This is the successor to one of the most popular open source revision control systems, CVS(Concurrent Versioning System). The developers of Subversion kept all of the good ideas that CVS had and removed the drawbacks. It’s been around since 2000, and still being actively maintained and updated. Major features are:
- Central repository – All your code is stored in one location
- Atomic Commits – If you commit multiple files, either they all go or they all fail. It’s not possible to have 5 out of 10 files commit to the system.
- Commit based revision system – Each commit increments the revision number, as opposed to one version number per file committed.
- Storage of file deltas – The source control system stores only the difference between files, rather than an entire new version of each file. This saves disk space. It also makes tagging and branching very cheap.
- Git/Mercurial - These could rightly be considered the successors to SVN. Git was created by Linus Torvalds, but is now maintained by others. Mercurial was the successor to BitKeeper, and both projects were started about the same time. While they are different products, I group them here because of their similarities.
- Distributed – Both of these source control systems throw out the notion that everything needs to be stored on one central server, like Subversion. The idea here is that there is a “Master” copy of the repository somewhere. However, should the Master copy fail for some reason, any others that have a copy of the filesystem can become the master copy. This is supposed to improve fault tolerance.
- Faster – SVN can occasionally take some time to do operations when revision history gets long. However, both Mercurial and Git scale better with larger codebases.
- Team Foundation Server – This is a Microsoft product that took a lot of the negatives associated with VSS (Visual Source Safe) and rewrote them in a better way. Microsoft added many of the ideas of the open source community, such as atomic commits, real branching and merging, and labeling (tagging).
Here are minor players that their zealots will swear by. I won’t take the space to do a full writeup. However, I’ve added links here in case you want to do your own research.
Here are revision control systems that you should strongly consider replacing as their drawbacks can be very costly.
- VSS – Microsoft’s first entry into the source control world. It’s disadvantages are checkout/checkin systems, no real branching, and the fact that your repository has a high percentage chance of becoming corrupt after it passes 2GB in size.
- CVS – Again, latest source control systems are 2 generations beyond this one. The longer you wait to make a change, the more painful the change will be.
- IBM Rational Clearcase – Ok, I have a personal vendetta against this one. It is difficult to use, has no real plugin for modern IDEs, doesn’t work half the time, requiring you to redo work, and you can’t recursively add files to the system (seriously?). I guess that’s enterprise software for you….
So, which one should you choose? As always, the answer depends. Here are some sample scenarios that you might look at.
- You don’t currently have a source control system, and your development staff is pretty inexperienced. You also need complete control. Your choice is Subversion. You manage the server, and it has one of the easiest learning curves out of all of them. There are numerous plugins for IDEs and you can pretty much use any development language you want.
- If you have a rock star development team that is already pretty familiar with revision control, you can get a ton of mileage out of Git or Mercurial. While I still believe your master copy should reside on a server, it doesn’t have to. Your rock star developers will appreciate not being held back by their development environment.
- You run a Microsoft shop and are heavily invested in MS technologies. You don’t mind paying for a good software product and want something that can integrate both development and deployment. Here, your choice is Team Foundation Server. Used correctly, it can significantly speed things up for you.
These are of course some very basic scenarios and hopefully they give you just a starting point for your research. As a manager, you need to balance features, cost, ease of setup and use, and your ROI. Be careful with anything you get from the sales staff of commercial products and try to do some research on what actual users of the products say. There are many blogs with comparisons of different source control systems out there. Above all, if you have rock star developers, trust their recommendation.
Lastly, there are some alternatives to running your own server. There are cloud-based instances of almost every source control product if you look hard enough. Two quick examples that come to mind (full disclosure, I have used neither of these products and cannot recommend or caution against) are Github, for open source projects and Bitbucket.
I sit in many groups on LinkedIn and see a lot of job postings go whizzing by. Some of the more popular groups that I’m part of have 10-20 new job postings daily, which seems like a lot. Many of these job postings are calling for Jr developers and I have to ask myself the question. Why?
I know, small and large business owners, that this is a tough economy and you are trying to save some money. You may even consider interns for a lot of your development needs. However, lets discuss this in some abstract terms so we can look at the situation correctly.
When a Jr developer (in skill, not necessarily title) is building an application, the effort is akin to throwing some cooked spaghetti against the wall and seeing what sticks. The developer can think of many ways to try to do something. If one of those happens to work, that is what ends up in your application. Your Jr developer is just happy because something worked. The items that are not taken into consideration are what the development effort might to do the network, database, computer running the program, or architecture. I like to use home building analogies when I talk about development, so let’s go with this. It would be like a home builder not worrying about how the electricity, gas or water will get into the house, not worrying about whether or not the house is situated properly on the plot or will interfere with the placement of a future road.
Unfortunately, there’s additional consideration with developing that you generally don’t have with building houses. You like the product you have, or your customers like the product you have, but you’ve gathered a few small suggestions for improvement. In short, you’d like to add on. This is perfectly normal, and developers would probably wonder what was wrong with you if you didn’t want the product to continue to evolve. If you’ve had a Jr heading up the building process, you’re going to find out you’ve incurred a lot of technical debt in the building process and that adding on is now cost prohibitive or impossible. When the Jr simply cannot do what you ask, you have to bring in someone more senior to take a look at the problem. This Sr will probably see such a mess that they’ll recommend a rebuild. It’s no different than having to rebuild the house because you need to re-orient it on the lot or make a major adjustment to the foundation to allow for the utility lines.
I know what you’re thinking – this guy has railed on Jr developers a lot and they really don’t deserve that. Besides, how are we going to get Sr Developers and System Architects if we don’t start out with some Jr Developers?
I completely agree. Let’s keep our technological pipeline full. However, this is a situation that more college or university coursework cannot rectify. Your Jr needs experience and the only way to get it is to try and fail and try again. I think what more workplaces need to make their Jr’s more effective is a decent mentoring program. It should include these things:
- Pair programming. Your Jr needs to sit with a smart and effective Sr or Architect and see how they solve problems.
- Architect the work out beforehand. This is probably as effective as pair programming. Your Architect plans out in UML how the application should work. Of course, your Architect cannot and should not plan out everything to the most minute detail. However, by seeing this example, your Jr will be able to get the overall idea behind an application structure and learn how they might solve the smaller problems in the app. Of course, your Architect will need to be available for questions.
- Code Reviews. After your Jr builds something, have a code review that focuses on growth. It’ll be discouraging if you mention everything that’s wrong, but pick some of the larger items and offer alternative ways of doing things. You might also want to consider having the Jr fix the items in the code review before declaring the work fit.
There are reasons to have Jr Developers in the office.
- You want to keep your pipeline full, realizing that there is a lot of turnover in the industry.
- You want to train them in your development methodology or your specific and obscure programming languages.
- You have a great hierarchy of upper level programmers that can mentor your Jr’s and bring them up to speed quickly.
- You’re planning a throw-away project. You need something to work for the short term. However, please realize that a Jr can turn a short term project into a long term project quickly. Also, most projects live long after their planned lifetime, so use this reason with caution.
The #1 reason not to bring a Jr developer into your organization is to save money. If you can’t provide this kind of infrastructure and development effort to your Jr developers, it’s probably not cost effective to hire them. 3 Jr’s at 35k a year will be less profitable than your 105k/year super effective Architect.
For additional reading, I’ve been really impressed with Grant Skinner’s Creating Great Developers series.
First, a quick primer on source control or version control systems (VCS) or revision control. A version control system is a piece of software that allows for the storage of multiple versions of files. Here’s a hypothetical situation:
Often in the development world when we’re learning a new programming language, the first application we build in that new language is a program called “Hello World.” It usually consists of one file and outputs the words “Hello World” to the screen or console. Say I started to learn Scala, and started with a Hello World application. I build out this one file and commit it to my version control system. I then decide I want to do some experimenting and I modify my Hello World file to the point where it will no longer compile. Using my version control system, I can revert my file to the previous version and begin again with a pristine Hello World file. From this brief example, I’m sure you can begin to see some advantages of working this way. We’ll go over more of them further down the article.
There are many development shops around that do not have adequate source control systems in place for their developers. I’ve even seen Fortune 500 companies that have large in-house development shops where this is the case. I can only think of a few reasons why this might be.
- The cost of moving away from a poor version control system is prohibitive.
- The politics/bureaucracy/red tape that must be gone through to make the change are difficult.
- The business case for a good version control system has not yet been made.
Whatever the reason might be for not having one, I’m going to propose that if you or your company is developing software, a modern source control system will save you time and money (and frustration, and possibly even developer turnover, etc). This applies even down to the one-man entrepreneurial software development shop. Here are some advantages to a modern version control system:
- You can revert code. If you are diligently committing to your source control system, you can revert your codebase to any commit point in the past. This has helped me many times. Here are some possible scenarios:
- The code worked fine one day, but not the next. You need to revert to a working version to make the client happy.
- The client wants to go back and pull some assets from an old version of the site for new work.
- Your server admins say that someone broke the build when they pushed the code to the QA server. You need to find out when the change was made and by whom.
- You can share code, easily, among as many developers as necessary, across multiple geographic locations. Even as few as two developers trying to share code across a network drive could easily waste an hour a day while they try to manage the code and not overwrite eachother’s changes. However, with version control, one developer commits, the other developer pulls down those changes and continues to work. Sometimes a code merge may be necessary, but it is highly preferable to the alternative. The time and costs of having more than two developers on a project without version control exponentially increase with each developer added.
- You avoid multiple versions of the same file or project on the network share. I’m sure we’ve all seen presentation.ppt, presentationfinal.ppt, presentationfinalrobertmodified.ppt, presentationfinalrobertmodified1017.ppt and similar on a network share. I’ve been guilty of doing it myself when there was no good version control system available. With source control, you have one file, and if you need to revert it to another version, it’s all there for you. No 12 versions of a similar file sitting there confusing everyone.
- You can track down bugs more easily. If a bug was introduced in one version of the software, you can go back in time, see which files were updated when the bug appeared, and even check the differences in individual files. This is highly preferable to making phone calls to individual developers who you think might have caused the bug and asking them what files were changed. Source control remembers it all.
- Most of the good ones are open source. You can download, modify, update, change, and use the software however you want. As long as you don’t expect support, these products cost only the bandwidth of the initial download, the server to run them, and the admin to maintain them. This is not a new software idea, and the disadvantages that older systems had are all worked out with modern systems.
In the next part of my series on source control, we’ll get out of the theoretical into stuff that can really be applied. We’ll define what I’ve been calling “modern source control” as concrete products. We’ll go over a couple scenarios and make recommendations that will save you time, money, and most definitely, somewhere, sometime, your bacon.
After 6-8 years reading other blogs and generally being an internet lurker, I’ve decided I have something to say and am taking the plunge into writing my own blog.
You can read more about me when I get the page up, but to give you a brief idea of where I’m coming from, here’s a small blurb. I’ve been in IT for all of my career, spanning about 15 years at this point. I started out in phone support and have worked my way through the IT hierarchy. I’ve done helpdesk, operations, systems administration, some limited networking, and end to end web development. One other tidbit about me is that I am an INTJ. I never had much faith in personality tests until I read about myself there, but it seems to fit me to a T.
Judging by the fact that I was able to get this domain name so easily, and based on my limited development experience at 4 or 5 different companies, I believe the software development process is something that has been ignored for far too long in too many companies/industries. The reason that the software development process isn’t scrutinized in many companies the way other processes might be scrutinized is the difficulty in understanding what software developers do.
Unfortunately, a bad development process causes many undesirable outcomes:
- Technical Debt.
- Wasted hours. I’ve seen quite a few companies unwilling to budget for a particular piece in the development process, only to spend well over that amount in developer hours wasted. For some reason, it seems that many managers treat their salaried developers as a fixed expense, rather than something they’re paying hourly for. If you can save hours, you can save money.
- Wasted time. This is not the same as above. All projects take a certain amount of time to complete. This is obviously variable based upon your development setup and the skill of your developers. However, if there’s a deadline to hit, wasted time can never be returned.
- The above 2 will lead to lower ROI. Every manager in your company understands that term, and nobody likes a low ROI. It’s important to prove that your software project will somehow help the company save money or time and be worth the investment.
Obviously, a good development process will cause the opposite of the above outcomes and endear you to your company management and possibly get you promoted (unless there are other political factors at play. For the sake of this article, let’s assume a perfect world, where good decisions are rewarded and unicorns run through the forest).
I hope to write regularly in this blog about the software development workflow, software that can improve your efficiency, hr and recruiting issues, project management, keeping your customers happy, and anything else the comes into my brain. This will be from the perspective of a web developer, so I hope to get comments from my desktop, embedded software, and mobile development friends about how things may be different or the same for them. Also, I’m open to suggestions for article improvement and new articles.