FBI ends faltering effort to overhaul software | CNET News.com

FBI ends faltering effort to overhaul software | CNET News.com

Yet another high profile software engineering project failure. I look forward to the details when they come out.

ACM Queue - Too Darned Big to Test - Testing large systems is a daunting task, but there are steps we can take to ease the pain.

ACM Queue - Too Darned Big to Test - Testing large systems is a daunting task, but there are steps we can take to ease the pain.

I liked this article on testing. It makes a few good points and makes people aware of some useful testing concerns and approaches.

Here is a short list of the things that I think are key to good quality assurance:

• QA is more than testing. Say that 10 times.


• You cannot test-in quality, you need to build-in quality. Say that 10 times.


• Start with high quality requirements. Review requirements. Demand specifics. Apply relevant knowledge of the domain.


• Have an actual QA plan document that sets quality goals, chooses appropriate strategies, and allocates resources.


• Design for quality and testability. The architecture must separate core business objects from the UI. In general, to aid testability, you separate components based on testing technique. Unit tests work well for logic-only business objects. System tests work better for UI interactions.


• Design for test access. It should be possible to construct a business object in any particular valid state, perform one operation on the object, and then verify the correctness of its state. This may require exposing additional methods to the test code that you would not want to expose to other components of the system.


• Build in tests in the form of input validation, assertions, preconditions, loop invariants, class invariants, and postconditions. Even if much of this is never executed, the process of writing these conditions and reviewing them greatly improves quality.


• Review the design and code. This helps catch errors and familiarize everyone on the team with the implementation. Do a little bit at a time. Build up checklists of common errors.


• Static analysis tools help detect common errors. More powerful analysis tools can do proofs.


• Unit tests should be written by developers at the time of writing each unit. Measure code coverage and argue for specification coverage. E.g., show that you cover boundary conditions and equivelence classes. When possible, use simple test oracles rather than hard-coded expected answers. Run unit tests very frequently, e.g., every night.


• Integration testing: this is not my favorate because it requires extra work. If possible, try to minimize this type of testing by simplifying interfaces and raising component quality. Try to combine a few components at a time in a test harness. Cover each configuration parameter and try all pairs of 2 configuration parameters, if practical. Fall back to integration testing when system testing indicates problems.


• System testing: test the entire system end-to-end for a given use case from the user's perspective. This is a good time to think about validation again. UI testing automation can help.


• Staging: test the system in a realistic setting with real customer data on hardware and networks that are identical to the production environment.


• Beta testing with key customers using real data in a real usage setting.


• Field failure reports capture detailed information about failures in deployed production software.


• Defect tracking: enumerate the defects and track them to resolution. Every defect gets tracked.


• SPI / Quality management: look back on defect reports and review meeting notes to find patterns of defects. Add additional checklist items to review checklists, and refine standards for design and implementation. Each defect could be the tip of an iceberg, look for related defects that share the same underlying error in developer understanding or the same symptom. Track testing coverage and continue to raise the expectation of coverage.

Forbes.com: The Collaborator

Forbes.com: The Collaborator

I'm glad to finally see CollabNet get more recognition. Hopefully more corporations will decided to sign up. I think that distributed development is really the most important software engineering trend of the 2000's.

The Apache Struts Web Application Framework

The Apache Struts Web Application Framework

• I like the simple, concrete forms-based approach
  
• I like the fact that they do not try to solve every concern,
  e.g., persistence
  
• The concept of actions and forwarding is a little too abstract and
  over generalized, however the conventional usage of these seems to
  be clear and concrete enough
  
• I like the option for client-side validation
  
• I like tiles. Would be even better if each tile was well-formed XML.
  
• I like the separation of syntactic validation in the ActionForm
  from semantic validation in the Action.
  
• The names ActionForm and ActionMapping are confusing.
  
• The name="" attribute in should be form="".
  
• The declaritive validator rules seem really awkward
  
• I like the separation of form, action, and page implementation from the
  struts-config.xml file. I think that can lead to reuse and fits
  the diverse skills of the implementation team.
  
• I like the replacement for HTML forms. HTML forms markup sucks.
  Might make it easier to transition to WML, XForms, etc. later.
  
• I like the support for internationalization
  
• The pages with taglibs are more verbose than Velocity was at
  CollabNet, but that is ok. They are well-formed XML.
  
• I like the fact that users cannot directly access .jsp files,
  especially since many jsp files have preconditions to be met.
  
• Tiles could be more cool if they leveraged a directory structure,
  like the way that zope does.
  
• I like the fact that forms and actions are separate. That should
  allow you to separate accessors from mutators at the webapp level
  and make claims about the entire set. E.g., no accessor has any
  side-effect.
  
• Role-based authorization are too simple, but OK for most uses.
  

Google Maps

Google Maps

The google maps site is very impressive. Here is my analysis of why.

1. It looks nice. The anti-aliased rendering and clear text names are superior to maps.yahoo.com and other maps services. That may seem simple-minded, but the whole reason to have a map is to ACTUALLY READ it, so it does matter.

2. Scrolling really matters. Maps.google.com is a great illustration of a very simple UI guideline that really matters. People have limited short-term memory and using tools can put a big load on limited human cognitive resources. E.g., after scrolling on maps.yahoo.com, the user must take the time to figure out what they are looking at by comparing map features to their short term memory of the map fragment shown before. Eliminating that user cognitive task makes maps.google.com a joy to use.

maps.google.com fits Ben Schneiderman's definition of direct manipulation:

• Continuous representation of the objects and actions of interest with meaningful visual metaphors


• Physical actions or presses of labeled buttons, instead of complex syntax


• Rapid incremental revsersible operations whiose effect on the object of interst is visible immediately

The concept of direct manipulation is not new, but it is refreshing to see it done so well in a web application. Virtually no other web applications make any attempt at direct manipulation. One exception is the content customization page of my.excite.com.

When it comes to searching, google keeps its one-text-box format and introduces a simple syntax. This is great for power users, but I think that they might be better off offering a more widget-oriented interface for occasional users.

3. The implementation of maps.google.com shows huge potential. Basically, its architecture is a rich client backed by a huge cluser with huge computing and storage power. I can think of many many uses for this type of system, I am sure that google has more coming.

FogBugz - Why FogBugz Works

FogBugz - Why FogBugz Works

The arguments in favor of fogbugz are pretty simple. Basically, it boils down to the claim that the fogbugz application is simple, and that professional developers naturally use any application that is simple. Not very insightful.

On the positive side, the UI is clean and their seems to be an integration with popular version control systems.

AdSense

I'm trying out adsense today. I have a had a really good experience with adwords, so I am looking forward to using adsense. I think that seeing the ads that appear in my blog will help motivate me to post more often as I review software engineering tools.

ReadySET Pro is now online and ready for sale.
It includes use cases, test cases, project proposals, qa plans, and a lot or other software process improvement.

I gave out two invitations already.