Choosing best SCM Tool – SCM Tool Features

**adsense_4x1Block**
Recalling the discussion of Goals of SCM will clearly paint a picture of what basic features an ideal SCM Tool should have in it.

• First and most basic functionality a Software Configuration Management tool should provide is support for a central file repository. All other functionalities are around managing and tracking of these repository items.

• Secondly it is important that the SCM tool provides the capabilities of the distributed team to work together from a central repository. Features like file locking mechanism, file comparison and atomic commits are to name a few.

• The SCM tool should also provide a simple mechanism for creating and maintaining private branches and for merging changes from the main code line to the private branch, and vice versa.

• The SCM tool should provide visibility into changes made for each task and support the ability to work by task instead of by individual file, to merge changes from one configuration to another, and to revert changes for a task if needed.

• The SCM tool should also provide an easy mechanism for rolling back to the last good integration version.

• The SCM tool supports simple creation of a hierarchy, give visibility into the changes at each stage, and enable straightforward merging between stages.

• Tagging is another feature very common and useful which involves giving meaningful names to specific revisions. These names are generally called Tags or Labels.

• The SCM tool should support re-targeting features without the need to write and maintain scripts to perform the operations.

• In order to re-factor code and still be able to trace through the history of changes, an SCM tool must support file and directory rename and move operations and track the operations as part of the element’s history.

• The SCM tool should easily integrate with the continuous integration server so that latest code from the SCM repository can be extracted and compiled continuously and whole process can be automated.

With SCM field maturing every day, more and more advanced features are introduced into these SCM Tools thus facilitating the job of a Software Configuration Manager and providing more control over the code base. By comparing these basic and advanced features you can choose best suited SCM tool for your business needs.

What other basic features you believe are integral part of every SCM Tool out there?

Configuration Manager’s Job Requirements

**adsense_4x1Block**
Achieving SCM objectives through day to day interactions with all departments of an organization need specialized skills comprising of both Technical and Personal skills.

However sometimes It is hard to find a resource having all these skills, thus companies need to provide proper training and guidance to existing employees in development, quality assurance or operations to take up on this responsibility.

Technical Skills:
Understanding of SCM Concepts: Thorough understanding of configuration management concepts is required by Configuration Managers so that they can work towards the desired goal with clear conscience. Understanding these concepts and goals will help them design policies and processes that won’t hinder someone else’s work while achieving desired results.

Experience with Code Repositories: Code repositories are heart of Configuration Management, a manager must be familiar with available code repository solutions and have working knowledge of one that is being used by his organization. Knowledge of common functions related to Code repositories including coding check-in, check-out , branching and tagging is mandatory.

Knowledge of Diverse Programming Languages: In today’s cross platform world there are rarely products supporting single platform. All that diversity pushes software houses to develop solutions in multiple programming languages and targeting multiple device types. To work along with this diversity, Configuration Managers should have working knowledge of diverse programming languages including Java, .Net, Objective-C, Ruby, PHP and ActionScript so that they can create build scripts for these different project types.

Cross Platform Knowledge: Products targeting large customer base needs to address every platform out there including Windows, Linux distributions and Mac OS. Knowledge of these operating systems and their command line languages will enable configuration managers to utilize their powers to maximum.

Continuous Integration Servers: Multiple and Diverse project types make it impossible for Configuration Managers to keep track of every aspect of a changing software. Thus knowledge and working experience of continuous integration servers is a must have.

Test Automation & Unit Test Integration: Creating unit and integration tests requires some degree of programming and testing skills, which a top configuration manager must possess to fully accomplish his job responsibilities.

Personality Profile:

Communication: The most important personal skill required in Configuration Managers is their ability to communicate with other departments. Their interaction must have all the ingredients of an effective communication to avoid any faults generated due to miscommunications alone.

Responsibility: Playing a central role in the product delivery work-flow demands responsibility to ensure timely delivery of the releases and in case of a problem it must be communicated to higher ups with due diligence.

Pressure: Dealing with deadlines has always been pressurizing, and Configuration Managers have to deal with deadlines when they have to release a product. CMs must be strong nerved and able to respond positively even in most adverse scenarios.

What other skills you believe are needed to be a good configuration manager?

Configuration Manager’s Job Description

**adsense_4x1Block**
Configuration Managers are hired and tasked with achieving goals set for a Software Configuration Management Department. Achieving those goals demands clear understanding of what should be done(job description) and what skills are required to do it (job requirements).

Job Description:

Understanding of SCM & Goals: It is very important for a Configuration Manager to know and understand software configuration management concepts, goals, benefits and standard strategies to achieve those goals. With thorough understanding managers will be able to develop SCM policies for their organization which won’t compromise the desired benefits.

SC Managers must also keep themselves abreast with latest industry developments in the domain of software configuration management and keep an eye on how others are doing it.

Liaison with Project Managers: In small organizations, Project Managers are usually taking care of some of the additional responsibilities of a Configuration Manager. However in medium to large organizations, where two individuals are assigned on these roles, a communication channel must remain opened between the two roles and close coordination is mandatory.

Irrespective of who makes the call on decisions pertaining to mainline management, branching, build cycles and release dates, both managers must always be on the same page on these subjects.

Liaison with Software Developers: Whoever is responsible for taking out functions of a Configuration Manager is required on daily basis to coordinate with developers on matters related to code check-ins, change tracking, build failures and communicating results of code analysis.

Liaison with Quality Assurance: Quality Assurance department receives builds generated by Configuration Managers. With more automation tools available, it is becoming convenient to automate unit and integration tests that are executed just before builds are handed over to Quality Assurance. Configuration Managers have to build a liaison with quality teams on improving test coverage of automated tests and staging releases as they are tested by QA.

Liaison with other Departments: Configuration Managers also interact with managers in other departments including Operations and Support. Operations department receives a fully tested build from Configuration Managers and deploy it on production environments. Support teams need to communicate with CMs when some issues need to be back tracked into time.

What you want from your Configuration Managers?

Pitfalls of Software Configuration Management

**adsense_4x1Block**
When implementing Software Configuration Management processes in an organization, one must be aware of common pitfalls that some configuration managers fall into. As rightly said, excess of everything is bad, so is true for SCM. We must ensure that while targeting goals and benefits of SCM we may not fall prey to these bad practices.

Process Gets in the Way:
One of the major caveat of implementing SCM processes is that they become so fat and start hindering activities of other departments.

Three common scenarios when process gets in the way are:

    a) When SCM policies start hindering activities of developers. SCM should support and speed up development rather than slowing them down. SCM managers must discuss with developers off and on to identify policies that are hindering their normal working flow so that those practices can be modified or abandoned altogether.
    b) Sometimes pre-check-in testing takes too long thus resulting in idle time of developers which is never desirable for any organization. Pre-check-in testing should be automated and divided so that most critical areas are targeted frequently. Less critical tests can be scheduled with a delayed cycle to speed-up development.
    c) Code freeze is a term used for a point in software development life cycle, before a release, after which developers are not encouraged to check-in any code into planned release source code. Sometimes configuration managers make this period so extended that developers keep waiting for checking-in their codes and thus resulting in idle resources. SCM processes must minimize the Code freeze period and implement processes that would allow developers to freely continue with their activities.

Long Integration Times at Project Release:
Having less frequent code integrations during an iteration may lead into long integration times when project is to be released. Specially if something is found broken, “Fixing it” in integration may end up missing a deadline and business.
SCM managers must ensure frequent integrations and implement policies that will make integrations smoother.

Inflexible Tools:
Configuration Management is there to help, rather than imposing its own limitations. Tools used for SCM must be selected while keeping all the goals and pitfalls of SCM in mind so that we do not end up serving the tools, rather than serving our own goals.

Lack of Expertise:
Coincide to Inflexible tools is inflexible configuration managers, as they cannot adapt as per organizational needs due to their own technical limitations. To fully benefit from SCM practices, proper technical training should be provided to people implementing SCM policies.

Staff Resistance:
If a company is new to SCM, then for right reasons SCM managers face resistance from all levels of organization. Managers and Developers alike feel SCM practices as additional work when compared to their normal development activities. This behaviour leads to non-cooperation with SCM managers and thus failed SCM efforts.

It is the responsibility of SCM managers to communicate benefits of SCM to every organizational unit that is going to be affected with new policies. Everyone must be convinced that people implementing policies are aware of potential pitfalls of SCM and they would do their best to avoid them. Only with internal commitment can SCM practices be implemented and its goals achieved

Failing to avoid these pitfalls will result in obvious consequences including, but not limited to, frustrate developers, financial and time losses and angry customers. In pursuit of SCM benefits we must not fall prey to these common mistakes.

Your Thoughts?
Did you ever experience similar bad practices in your organization, or behaviors of your colleagues that would have hindered SCM implementation.

Goals & Benefits of Software Configuration Management

**adsense_4x1Block**
Let us go through some goals of SCM in general, which when achieved are transformed into benefits for your Organization.

Sandboxing with private build before check-in:
Rapid application development and frequent changes to source code demands frequent check-ins, it is important that the developer has some confidence that the code will successfully build and pass unit testing before he integrates it into the main code line. By utilizing private check-ins at an isolated area off the main line (e.g., a private branch) and building the code in the private sandbox before checking it into the main code line, the developer can test and debug the code locally, and only after passing the build and unit tests can the code be integrated into the main code line. The benefit of this SCM goal is that it decreases likelihood of a failed integration build.

Task-oriented development:
In today’s software organizations, that are adapting to Agile methodologies, application features are usually broken up into the smallest size tasks possible so that each task can be tracked, integrated, and debugged. One of the goals of SCM is to enable an organization to perform task-oriented development that includes providing visibility into changes made for each task, supporting the ability to work by task instead of by individual file, to merge changes from one configuration to another, and to revert changes for a task if needed. Biggest benefit of task oriented development is maximum utilization of resources.

Ability to revert to last good working version when integration testing fails:
Even though a private build prior to integrating changes reduces the chance of a broken build, there are still cases when an integration build will fail. For example, if two developers commit changes at about the same time, it is possible that each one did not test his individual changes with those of the other.

In the event of a build failure, it is a goal of SCM i to be able to revert to the last good build so that other developers have a clean configuration of code to use for their own work. Reverting to the last good configuration allows one engineer to fix the broken build while the rest of the team can continue working on their tasks and verifying the work with their own private builds.

Staging hierarchy:
Having all development work from the main line often causes chaos. This can be mitigated by use of a development hierarchy. A development hierarchy is simply a hierarchical representation of the dependencies between groups that includes process steps such as integration, quality assurance, and code reviews.

A separate code configuration is used for each stage in the hierarchy. It is a natural extension of private branching. As a change is pushed from one stage to the next, the particular change as well as the system as a whole reaches a higher level of maturity. Continuous multi-stage integration can be employed to automatically build and test changes as they are pushed to each stage.

Goal of SCM is to support the simple creation of a hierarchy, give visibility into the changes at each stage, and enable straightforward merging between stages. Utilizing a staging hierarchy along with private versioning gives the added benefit of more code stability at each step up the hierarchy.

Ability to revert and retarget changes:
Generally software requirements are driven by the business and can be changed throughout the SDLC. For this reason, it is the goal of SCM system to be able to identify the changes made for a particular feature and remove the changes from one iteration and retarget them to the next.

Refactoring support:
Agile development methods focus on a simple design for the features being developed during the current iteration and not on designing an architecture that will meet the needs of all features for the lifespan of the product.

For this reason, code must periodically be refactored to make it easier to understand and reusable for other features. Thus a goal of SCM to support refactoring of code and still be able to trace through the history of changes.

Geographically distributed development:
With technology it is possible today to utilize specialized talents that are even geographically distributed. Due to this distributed nature of software development organizations, it is not always possible to assemble cross-functional teams without involving team members at more than one location. For these scenarios, it is important that the SCM provides a central repository so that members of distributed teams can work together.

Continuous integration rather than large-scale integration phases at end of project:
One major goal of SCM is to enable continuous integration of changes. Each developer should submit changes as soon as a task is complete. Continuous integration build servers can be used to poll the code repository periodically and, if any changes exist, build the new configuration. Equally important to this concept is the immediate reporting of build status. If a build breaks or any of the unit tests fail, the developer who recently committed the changes must fix the code right away.

Your Experience?
What benefits of Software Configuration Management you have experienced for your organization?

Software Configuration Management in Your Organization

**adsense_4x1Block**
Every activity that contributes to the success of a project becomes integral part of the organization. Same is the case with Configuration Management. In every Software House today, SCM practices are applied in one form or the other. Thus we must understand where SCM activities fit in our organization, so that we can capitalize on them to further improve our productivity.

Activity Positioning

From activity perspective, Software Configuration Management can fairly be placed in the center of all project activities including Project Management, Development, Quality Assurance and Site Operations.

Software Configuration manager interacts with all of these different functional units of a software house, thus further emphasizing its importance in an organization.

General SCM Workflow

While keeping SCM definition in perspective, let’s go through the general workflow of Software Configuration Management activities that are carried out in a medium sized software house and how they are intertwined with Software Development activities.

SCM Workflow starts as soon as project management decides what new features are to be added to existing or new software in next iteration, depending upon whatever Software Development Methodology they are using.

Once new changes are decided, Developers start coding these changes into existing program by updating source code. They add new lines of code, add new third party components that are needed for new features, delete some redundant files and so on. At this stage, SCM ensures identification of new items that are added to the system. Controls access and permissions on these items between multiple developers, and records all changes applied to existing project source code.

While developers are coding these new functionalities, SCM requires that health of the changing source code is monitored at convenient intervals to make sure there are no surprises related to broken dependencies of software at later stages. In case there are some discrepancies, SCM ensures availability of change reports so that culprits can be identified.

Once development is complete, SCM further verify system state by executing unit and automated tests before handing the build to Quality Assurance for their consumption.

When QA has verified that current release is production ready, it again falls in the domain of configuration management, to configure the software product for prime time and hand it over to Site Operations, who deploy the product for end customers.

SCM Boundaries

As you might have noticed, boundaries of Software Configuration Management overlap with all other engineering departments to some extent. In organizations where proper SCM practices are not implemented explicitly, SCM activities are still carried out, however in this case, individuals in other departments step-up to share these responsibilities.

For example, in such an organization, Project Manager would be ensuring complete execution of the SCM workflow i.e. handing over builds from one department to other till they are deployed on production, manages project dependencies etc.

Developers will ensure there are no conflicts submitted to the project source code, no dependencies are left out on their local machine and even resolve conflicts or revert back some source files in case they are mistakenly overwritten.

Without a configuration manager, executing Unit and Automated tests becomes responsibility of Quality Assurance team. Thus they ensure quality of the product, right from developers’ hands.

Site Operations in this scenario would apply final configurations to the build and deploy it on the production server.

At the end, it all depends upon time, monetary and space resources each organization has on its disposal, which would decide whether they need a specialized individual to take up SCM responsibilities, or to groom existing resources from other departments to take up these additional responsibilities.

What is Software Configuration Management and its Functions

**adsense_4x1Block**
Configuration Management defined by IEEE is:

“Configuration Management is the process of identifying and defining the items in the system, controlling the change of these items throughout their lifecycle, recording and reporting the status of items and change requests, and verifying the completeness and correctness of items” – IEEE Std-729-1983

This definition covers the topic of Configuration Management in general, and can be applied to any system that is composed of individual items, undergoes change, and needs tracking and testing. For example a car assembly can be one such system.

However when put to Software context, Software Configuration Management as per above definition involves:

Identification of Items: Every single item that contributes in the successful creation and then execution of a software solution needs to be identified. An item can be anything including source code files, third party libraries, third party components, image and media resources, database scripts etc.

Controlling of Items: Once identified, Configuration Management ensures control over addition, update and deletion of these items so that critical components of a software system do not get missing when needed. Controlling also involves granting permissions and rights to individuals, for different actions that can be performed on these items individually (new lines of code in a file) or collectively (taking a snapshot of complete source code).

Recording of Actions: Every action performed on these items must be traceable in case something goes wrong and the complete software program when built does not behave as expected. For this purpose every action taken on system items, is recorded in some form, let it be a simple log file or managing extensive time line of changes.

Reporting of Changes: Recording alone is not sufficient; a Configuration Manager must be able to search through these recorded details whenever something needs to be reverted back to previous state. Thus reporting of every action performed on system items remains an integral part of Software Configuration Management.

Verifying System State: Last but not least, Configuration Management must ensure completeness, correctness and stability of complete software system at given point of time. This serves as an early warning system for software managers, so that they become aware of any discrepancies in constituting items, even before a build is released to Quality Assurance.

Above definition lists five Functions of Software Configuration Management. Any activity within an organization for managing software configurations will fall within one of these functions.

History of Software Configuration Management (SCM)

**adsense_4x1Block**
Software Configuration Management (SCM), simply put, involves everything related to management of evolving Software Projects’ Code. Software Configuration Management is today a mature software engineering discipline and essential part of every software development team, just like Quality Assurance. Let’s take a quick trip through evolution of SCM to understand why it was needed at first place.

History of Software Configuration Management (SCM)

The need for formalizing and taking SCM seriously, arose when rightly named “Software Crisis” was identified back in late 70s and early 80s. It was realized that mere software programming is not everything in Software Engineering (SE). There were other serious issues that were hindering software development including architecture of the software, source integration from multiple developers, building projects, incorporating ongoing changes and so on.

In late 70s and early 80s, Software development started taking first steps towards programming large software solutions. Thus very first concerns of SCM were versioning of incrementing software, rebuilding and managing composition of complex systems.

The first SCM systems of early 80s were developed in house and their primary focus was on file control. Most of them were built as a set of Unix scripts over RCS (a simple version control tool) and Make (for derived object control).

Some other worth mentioning tools of that period are; DSEE, the only serious commercial product, which introduced the system model concept which was an Architecture Description Language ancestor; NSE, which introduced workspace and cooperative work control; Adele, which introduced a specialized product model with automatic configuration building, and Aides de Camp (now TRUE software) which introduced the change set.

In early 90s, as software and hardware industry matured, software vendors became capable of developing many projects simultaneously. Thus SCM in that era addressed issues around process support for developing multiple projects at a time, and concurrent engineering.

The first real SCM products appeared in the early 90s. These systems were much better. They often used a relational database but still relied on file control, they provided workspace support, but no or built-in process support. This generation included Clear Case (DSEE successor) which introduced the virtual file system and Continues which introduced, with Adele, explicit process support. Continues and Clear Case are currently the market leaders.

In late 90s, with the advent of web, remote programming became viable thus gave rise to issues pertaining to management of code submitted by remote teams. SCM in that period expanded its scope to support programming in the web.

During this period process support was added and most products matured. This period saw the sanctification of SCM, as a mature, reliable and essential technology for successful software development; the SCM market was over $1 billion sales in 1998. Many observers consider SCM as one of the very few Software Engineering successes.

Post 2000 period brought further optimizations and innovations in the working of SCM systems and also brought new SCM solutions like Subversion with improved memory utilization, atomic commits and much more. Another relatively new but innovative entry in SCM domain is Git. Git introduced the innovation of treating every clone as a full-fledged repository with complete history and full revision tracking capabilities.