by Brad Hart
In just about every situation in the last 8 years where I’ve gone into a prospect to talk about replacing ClearCase, I’ve been asked the question about ClearCase’s Dynamic Views and why AccuRev does not have a similar concept. It’s a fair question coming from those who are familiar with ClearCase and I’m posting this blog to help both give some background information on Dynamic Views and answer some of the common issues raised by former users of ClearCase before they made the switch to AccuRev. I used to work at Rational Software in both Support and in the Field and I spent a number of years as a ClearCase consultant before coming to AccuRev in 2001.
At the time Dynamic views were introduced, there was tremendous pain in the market using a local source copy model, especially in enterprise applications. Disk space was extremely expensive, and it was becoming increasingly infeasible to have large enough disks on each developer’s workstation. Networks were also much slower, and the time required to copy entire sets of source code to each developer’s workstation was unrealistic as applications grew in size and complexity. Dynamic views provided the appearance of each developer having a local copy of the source files, but without the time / disk space overhead associated with having real local copies. They also provided just-in-time access to files across a network connection which was transparent to the end user, similar to the way NFS works. Unlike NFS, which you only can access the latest version of files, the dynamic views allow the developer to reconfigure their view of the files to represent any given configuration past or present. Also, unlike a local copy model, reconfiguring what a developer sees does not require any file copying to reflect the changes. This saves time and money and the savings continue to scale the larger the development group gets.
Does it still hold water?
No. Both workstation and network hardware costs have dramatically dropped in recent years, and the performance has increased exponentially. It is very common and reasonable for developers to have near server-class systems on their desktops. In many cases, it is now a much better time savings to have developers work with local copies of their source files. In fact, Rational’s default usage model for developers is to do their development in a local copy source file model, contradicting the presence of Dynamic views. Dynamic views were a time and cost savings breakthrough when it was introduced, but given the changes in development environments in the current time, it is more often than not seen as a hindrance. There is also a much higher administrative burden associated with Dynamic views. Especially if you are working in a mixed environment (SAMBA, TAS, etc… need to be properly configured and maintained). Also, Dynamic views are notoriously unreliable and unusable over remote connections. Another major objection to Dynamic views from the developer perspective is that most developers don’t want “the rug pulled out from under them.” Your files are constantly changing in your view….how are you supposed to develop/build and test like that? Add in the fact that ClearCase does not have atomic transactions, and developers using Dynamic views will constantly have inconsistent sets of code to work on. Bottom line is that even Rational recommends developers use Snapshot views (like AccuRev workspaces) and only use the Dynamic views for integrations. Since AccuRev truly builds in parallel development, you don’t need an integration view/workspace. All your work can be done directly from one workspace.
Five things I’ve heard from developers on why they think dynamic views are important to an effective development environment:
WYSIWYG: The final test of your code changes before check-in is exactly the same thing as testing the release area code directly. No need to “check it out again in a different place just to make sure I checked in everything right.”
AccuRev allows your private work (keeps) and your check-ins (promote) to all occur from the same place (you don’t have to check out to a different place). AccuRev builds in best-practices like private-branching (workspace streams), atomic transactions, and copy-merge. You don’t get that out of the box with ClearCase. AccuRev’s built-in best practices absolutely improve the entire process. You absolutely must merge against the latest code before you promote your changes (for overlapped files). Plus, developers have total control of their workspace bringing in new changes as they are ready. That way if something is broken, they will know whether it is their code, or the latest code from the mainline. With Dynamic views, you will have to go find out for yourself and it is constantly changing. I have heard a lot of the “rug being pulled out from under me” analogies regarding dynamic views.
Zero-latency perfect synchronization with the release area: no time is ever spent checking out all the files in the entire directory just to see the release area. You are always “seeing” the release area. Your code is the release area.
Running updates in AccuRev is trivial and incremental. Performance far exceeds ClearCase’s snapshot views. You don’t need to check out all the files. Updates just bring in the changes.
Instantaneous creation of a new view into the release area: Suppose you want to do a quick mod and test of the release area code. Just create a new view, make the code mod, and do a build. You are good to go. No time spent doing a full checkout of everything just to modify 1 line of code in 1 file.
With AccuRev, you can simply re-parent one of your workspaces, update, check in your changes…done.
Automatic synchronization with the continuous development of code in the release area. Suppose you have a huge pile of code checked out for 3 weeks, and one week into it somebody checked in a change that broke your implementation. Dynamic views force you to reconcile the merge difference the moment the file is checked in, while it is still fresh in everybody’s mind. No trying to recreate the scenario 2 weeks later because you forgot to do a refresh of your sandbox. Your build will immediately fail, and you can confront the guilty party before they go on to work on other stuff. With dynamic views, the team is always in alignment and working on the code together.
Too much thrashing. This might work for a very small team. This would be mayhem for a large group.
Entire directories can be rearranged without worrying about somebody’s checkout becoming unmanageable. Since everyone instantly adopts all the changes that take place in the release area, all the files in a directory can be moved and changed without you even being aware of it. All of your “checked out sandboxes” instantly are updated to work with the new directory structure. This freedom encourages engineers to create a well structured file hierarchy, even making changes during critical bugfix periods in a product development cycle a non-issue. One does not feel the need to keep around old sandboxes “just in case everything breaks.”
Absolutely no problem for AccuRev. AccuRev has true namespace versioning and handles all elements using element ids. You can work on a file and have someone else rename it and it will cause you no problems at all. When you update, the file will be moved to the correct location and your changes will remain in effect.
This post is not meant to discredit the work done by the ClearCase development team, but rather to point out how Dynamic views have all but become obsolete. The VCR changed home entertainment forever…but you’d have to pull my DVR and DVD players from my cold dead fingers while my VCR is nothing but a fond memory collecting dust in the basement.
What are your thoughts?
-Brad
The necessity of dynamic views due to the expense of disk space and computing power no longer holds water, but I don’t buy any of your reasons for it being unnecessary.
Developers don’t necessarily have to worry about a “rug” being pulled out from under them. Their config-specs can simply freeze on a time-basis, or they can work on their own branch(es) to whatever granularity they so choose. Alternatively, they can cherry-pick checkouts on an individual basis if needed.
Some developers even want the “rug” pulled out from under them because they are working on isolated parts of the system that won’t be disrupted by changes from outside of them. Speaking of “best practices,” systems setup to automatically label the last known “good” build also ameliorate the issue.
If performance is an issue for builds, snapshot views are also an option–created either by the native client or by the newer remote client (CCRC). So is alternating between both.
In general, comparing the performance of an SCM that only deals with synchronizing copies (probably everything _but_ ClearCase) to a dynamic view is akin to comparing the performance between a local file-system and a networked file-system in general. It doesn’t make sense. Further, given that I can nearly saturate the bandwidth of my NIC when creating a snapshot view, I hardly think performance is of any practical concern. Hence, I don’t think the claim that “Performance far exceeds ClearCase’s snapshot views” holds water either.
The rest of the “reasons” you provide follow a similar pattern–they are presumptive of the model being used for ClearCase.
Further, you don’t mention the (probably distinguishing) feature that dynamic views provide (via MVFS): build auditing. The ability to know exactly what went into a build can be absolutely essential. For example, companies that produce safety-critical software and devices, that actually care about fixing defects in products already deployed in the field, can use that information to answer the question: what products have ever used the particular file/version where we discovered a defect? Then, rebuild all the affected products with the newer fixed version.
ClearCase has a number of technical deficiencies. The only one I see mentioned here is the lack of “atomic transactions,” which pertains to atomic checkins of changesets (“activities” as they’re called in ClearCase parlance) that occur across VOBs–which makes sense–it’s difficult to guarantee a single atomic transaction that would occur in multiple databases.
Thanks for the reply Garen. You make some good points about ClearCase when used properly, but that is the key. Most people either don’t know how or can’t be bothered to use ClearCase properly. Unless they have a tools admin team writing automation on top for them, many folks just use ClearCase out of the box with very little process. Time rules & private branching are good practices that many people avoid due to the overhead. Use of those concepts certainly help when using dynamic views but of course would help even with snapshot views or any SCM system’s workspace model (I’m a big believer in private branching).
The whole point of UCM was to encourage a standard set of best practices on top of ClearCase (matter of fact it was called SUM – Standard Usage Model, internally before it was released in CC 4.0). There are many advanced users like yourself out there who know how to work ClearCase well, but the vast majority do not. Despite UCM’s well known problems, it was a good concept since most people were struggling with base ClearCase. So while you are correct that the negatives of dynamic views depend on the usage model, most people aren’t using them right.
In today’s world, the *only* saving grace of dynamic views is the build auditing as you mentioned. This also assumes you are willing to use ClearMake and assume the overhead in build peformance with examining objects for wink-in / sharing.
The benefits of build auditing can also be replaced using modern techniques and tools.
Continuous integration gives you a repeatable and reliable build process. Other stand-alone build tools such as OpenMake and Electric Cloud help correlate builds vs. source and when integrated with a rock solid SCM tool like AccuRev allows you to always know what sources when into each release & deployment artifacts.
Bottom line is the cost vs. benefit ratio of dynamic views isn’t what it used to be, especially since all the benefits can be easily replaced by modern tool & techniques that are easier and cheaper to implement.
“many folks just use ClearCase out of the box with very little process.”
While that may be true, you can’t talk abut the usefulness of of a CC feature under that assumption. As anyone who uses CC knows, using CC out of the box is not very effective.
Personally, I think the main issue with MVFS are performance and the added complexity of CC needed for file drivers per platform. Using a snapshot model (as all other SCM tools) makes it easier to run on multiple platforms etc.
But I agree that given the performance gains with local files versus network/MVFS snapshots can reduce the more more costly built/rebuild cycle. For example, we have devs using CC snapshot views and never use dynamic views because they do not use ClearMake and the build times are between 10 and 20 times faster.
We have another group that is using ClearMake and Winks in many objects and and the build audit is valuable. But when a dreaded global header files changes, so much for ClearMake shared objects… And if you change a header in your stream, you can never wink in again for the life of the stream which is simply mistake in the design of CC/Cmake.
So, while the Audit build can be very valuable there are many problems with dynamic views in the performance area. And as far as snapshots taking lots of bandwith, as long as the update model is incremental, that is only a one time cost per workspace so its not that bad day to day.
Where Accurev’s snapshot model really shines is in its flexibility. Developers using and CC can not stand (and with good reason) having to create (2) views in order to deliver code. As was stated, if you rebase your stream, you are getting 100% the same as the integ stream at the moment so why, just a few minues/hours later do you have to go through the whole process again. Yes, it would catch last minute integration issues but why the two views?
Seems to me that Accurev at least uses a single workspace that is able to deal with local checkins (keep) as well as do the final deliver (promote). In CC this is a much slower process due to the two-step deliver. Of course I am talking UCM CC not base.
Garen,
It is worth noting that while a CC snapshot view may saturate your NIC, snapshot views in CC are not the same as workspaces in AccuRev. CC was originally designed around views. In order for a snapshot view to calculate what has changed, it has to rely on the functionality of CC views. Calculating what has changed in a CC view is very slow. AccuRev was originally designed around workspaces. So, if only a couple of files have been changed, the server figures that out quickly and only sends those files. For CC to know that only a few files have changed, it needs to do a lot more work which is one of the reasons it saturates your NIC.
One example of how AccuRev can calculate differences quickly is that it is transaction based. So, if there have been no transactions at all since you last did an update, you know right away there are no changes.
Cheers!
Damon