Coder to Developer: Tools and Strategies for Delivering Your Software by Mike Gunderloy : Introduction and Chapter 1

Introduction

The author distinguishes between coders, who know syntax and semantics of computer language and developers, who can apply that knowledge to turning out a working application with all the supporting details. The book is targetted to help a coder become a developer.

The author then provides an outline of how the book is organized into a sensible order of chapters so that the reader completes the task to ship a successful software product although every chapter is organized as a stand alone one.

Chapter 1
The author emphasizes on requirements management and choosing a methodology before starting to write a single line of code. The author then provides a list of questions to do brainstorming to start software development process. The author then discusses two concepts:

• The elevator pitch: The idea was popularized in dot-com era when instead of 15-minute presentation, entrepreneurs were told to explain their ideas to a VC in an elevator ride to 87th floor in 30-seconds. Based on this, the author provides a rule to write a short statement describing what the software will do and why. The author warns the developer to refrain away from feature creep - the continuing addition of new functionality when the old ones are still being coded. The author provides some points to keep in mind while coming up with elevator pitch and creates one for his own software.
  
• The mind map: Mind-map is a form of diagram for free-form brainstorming. The author then provides a list of softwares to use mind-map on Windows. The author also provides mind-map of the software he is going to create in the book called Document Tracker.

The author then classifies the requirements in four sets:

• Business requirements: These are high-level objectives of the organization.
• User requirements: The goals the user must be able to perform.
  
• Functional requirements: Software functionality that must be built for the user to accomplish their tasks.
• System requirements:

The author then provides a checklist to help elicit requirements at the time of working with external customer. The author then discuss about ways to capture requirements and describes use case as one of them. The author then enlists use cases of the tool Document Tracker. The author then provides a list of items that should be present in the use case document.

Instead of focusing on major milestones, the author advises the developers and managers to focus on small tasks, e.g. instead of defining a target date for user interface, set target date for individual dialog boxes and forms. The author then provides some examples of project tracking and billing tools.

The author provides a list of issues in using Microsoft Project for software project management such as it is overkill, designed for huge software projects etc... The author then talks about different methodologies of software development such as:

1. Waterfall Approach: The method is like a waterfall which starts from requirements analysis and then flow downhill through preliminary design, detailed design, implementation and system testing and acceptance testing.
2. Iterative Approach: All the project phases like development testing and planning are repeated many times in the course of the project. The author provides two examples Rational Unified Process (RUP) and Microsoft Solutions Framework (MSF) of iterative approaches.
   
3. Agile Approach: The author lists methodologies such as Extreme Programming (XP), Scrum, and Crystal for rapid application development.

The author finally recommends to use what works and leave the rest. Both waterfall and iterative approaches measure the cost of an entire project upfront while agile methodologies keep delivering small chunks until time or money runs out. The author also mentions test-driven-development method which resulted from extreme programming.

The author then provides a checklist while planning the software to make sure you are building the right thing.

The Art of Software Testing by Glenford Myers : My Opinion

I would give this book 5 out of 5. It is a must read book for both programmers and quality assurance engineers. It should be included in colleges as a textbook along with Code Complete that I described earlier.
To make it easy to browse, I am putting links to all the chapters below:

• Preface
• Chapter 1
• Chapter 2
• Chapter 3
• Chapter 4
• Chapter 5
• Chapter 6
• Chapter 7
• Chapter 8

The Art of Software Testing by Glenford Myers : Chapter 8

The author discusses about testing tools and other techniques in this chapter. The test case design has not been automated but significant advancements have been made in automating other aspects of the testing process. Some of the tools are:

• Module driver tools: It provides a language to express test cases to test a module. The author mentions three advantages of such tool:
1. The language reduces the effort required and standardizes the form of test cases.
2. Rerun of test cases is easy when a correction is made to the module
3. The automatic verification of results forces the programmer to state explicitly the expected output.
• The author mentions AUT (Automated Unit Test), MTS and TESTMASTER as examples.
  
• Static flow analysis tools: They analyze control flow and data flow of the program, they can detect following errors:
• Variable with an undefined value
• Variables with assigned values that are never used
• Unreachable code
• Inconsistent interfaces among modules
• ACES, FACES, AUDIT, RXVP and DAVE are examples of this kind of tools.
• Test coverage monitors: Coverage monitors are tools that monitor the subject program during execution to provide statistics to determine whether the statement-coverage and decision-coverage criteria have been met.
• Mathematical proof of program correctness: The proof of program correctness tries to demonstrate that the program is error free. The author provides references to method of inductive assertions, predicate transformers, subgoal induction, computation induction, structural induction and intermittent assertions.
• Program correctness provers: The goal is to create a program that, takes as input another program and its input and output assertions, attempt to prove the subject program correct.
• Symbolic Execution: It is a hybrid between program testing and correctness provers. The subject program is executed symbolically. 
• Test Data Generators: The author talks about several tools that generate test data to meet a specific criterion by analyzing a program's logic flow.
• Environmental Simulators: These are useful when testing in actual environment cannot be done, test conditions are hard to create in real environment etc...
• Sneak Circuit Analysis: 
• Virtual Machines: They provide multiple programs impression that each is executing in separate standalone machine.
• Testing Mathematical Software: 
• Interactive Debugging Tools: The author discuss about EXDAMS, ISMS and CAPS.
• Compiler Debugging Aids: Compilers can contain debugging features added to the language such as
• Ability to designate any arbitrary statements as debugging statements and to activate and deactivate at compilation or run time.
  
• Program State Monitors: It monitors a program during execution, watching for the arrival of a certain predefined conditions and/or collecting data that would be useful during debugging such as first, last, minimum and maximum of each assignment statement, minimum and maximum values of each do-loop control variable etc...
  

The author then discusses about software error studies, software error data-collection and predictive models.  The author then provides some statistics on testing experiments and the results. The author then provides some findings on debugging experiments.

The author discusses how computer architecture or hardware design can help debugging by providing features not present in the current computer design.

The Art of Software Testing by Glenford Myers : Chapter 7

Debugging is defined as determining the exact nature and location of the suspected error within the program and fixing or repairing the error.

The author then discusses debugging methods:

1. Brute-force debugging: It is the most inefficient and unsuccessful method and advises to use this only when all other method fails. The author discusses three methods of brute-force debugging.
• Debugging with a storage dump: This is the most efficient method and the author provides a list of problems associated with this method including static nature of the dump, massive amounts of irrelevant data, etc...
  
• Debugging with scattered print statements: The author also enlists the shortcomings of this technique such as massive amount of data, require to change program, etc...
• Debugging with automated debugging tools: The author compares debugging with solving a homicide and emphasizes the use of brain over a set of aids.
  
2. Debugging by Induction: The author provides steps in the induction process as follows:
• Locate the pertinent data
• Organize the data
• Find relationship among data
• Devise a hypothesis
• Prove the hypothesis
3. Debugging by Deduction: The author provides steps in the deduction process as follows:
• Enumerate the possible causes
• Use the data to eliminate the possible causes
• Refine the remaining hypothesis
• Prove the remaining hypothesis
4. Debugging by Backtracking: In this method, mental reverse execution of the program is performed in which values of variables and state of the program is deduced by looking at the wrong output and the process is continued till the origin of the first unexpected state.
   
5. Debugging by Testing: Separate set of "slim" test cases are developed to provide information useful in locating a suspected error.

The author then provides two set of principles:

1. Error locating principles
2. Error repairing principles
• When there is one bug, there is likely to be another.
• Fix the error, not just the symptom of it.
• The probability of fix being correct is not 100%.
• The probability of the fix being correct drops as the size of the program grows.
• Beware of the possibility that an error correction creates a new error.
• The process of error repair should put one back temporarily in design phase.
• Change the source code, not the object code.

The author then discusses about improvement that can be achieved by careful analysis of software errors:

1. When was the error made?
2. Who made the error?
3. What was done incorrectly?
4. How could the error have be prevented?
5. Why wasn't the error detected earlier?
6. How could the error have been detected earlier?
7. How was the error found?

The Art of Software Testing by Glenford Myers : Chapter 6

In this chapter, the author focuses on the part of testing that starts when human-testing ends. The author talks about origination of errors in software programming. Since software development is a process of communicating information about the program and translating it from one form to another, the vast majority of software errors are attributable to breakdowns, noise and mistakes in communication. The author then describes the flow of the software development process. The author suggests to include more precision into the development process and having a separate verification step at the end of each process. The author also suggests having distinct testing processes toward distinct development processes i.e. focus each testing process on a particular translation step. The author then discusses testing at different stage of development cycle:

1. Function Testing: It is a process of finding discrepancies between the program and its external specification.
2. System Testing:
• System testing is the process of attempting to demonstrate how the program does not meet its objectives.
• System testing is impossible if the project has not produced a written set of measurable objectives for its product.
• The author then mentions following categories of test cases:
• Facility Testing: It is the determination of whether each facility mentioned in the objectives was actually implemented.
  
• Volume Testing: The program is subjected to heavy volume of data.
• Stress Testing: It involves testing a program under heavy load in a short span of time.
• Usability Testing: The author provides following aspects of usability testing
  
• Does system contain excessive number of options ?
• Does system return acknowledgment to all inputs?
• Is the program easy to use?
• Where accuracy is vital, is sufficient redundancy present?
• Are the outputs of the program meaningful?
• Security Testing: It is a process of attempting to devise test cases to subvert the program's security checks.
• Performance Testing: Test cases should be designed to show that the program does not meet its performance objectives like response times and throughput rates.
• Storage Testing: Test cases should be designed to show that the program does not meet storage objectives in terms of main and secondary storage and spill/temporary files.
• Configuration Testing: The program should be tested with each kind of hardware device and with the minimum and maximum configuration. Each possible configuration of the program should also be tested.
• Compatibility/Conversion Testing: Test cases should be designed to make sure the program meets the objectives of compatibility with and conversion procedures from, the existing system.
• Installability Testing: Testing of installation procedures should be part of this testing.
• Reliability Testing: If the objective of the program contain specific statements about reliability, tests should be devised to
• Recovery Testing: Tests should be designed to see the system's recovery from programming errors, hardware failures and data errors.
• Serviceability Testing: Serviceability objectives defined in terms of service aids, mean time to debug a problem, maintenance procedures, and quality of internal-logic documentation.
• Documentation Testing: User documentation should be subject to an inspection for accuracy and clarity. Examples in the documentation should be part of test cases used to test the program.
• Procedure Testing: Any human procedures involved in large programs should be tested.
• The author clearly specifies not performing system test by programmers who have written the program as well as not within the organization developing the program.
  
3. Acceptance Testing: This is carried out by conducting tests to show that the program does not do what it is contracted to do.
4. Installation Testing: This may include test cases to check that all the files have been created and have necessary contents, all parts of the system exists and is working.

The author then discusses about test planning and enlists following components of a good test plan:

1. Objectives: Each testing phase should have an objective.
2. Completion criteria: Criteria for specifying the completion of each test phase needs to be specified.
   
3. Schedules: The schedules for when the test cases will be designed, written and executed should be created.
4. Responsibilities: The responsibilities of people regarding testing and fixing of errors should be clearly defined.
   
5. Test-case libraries and standards: Systematic methods of identifying, writing and storing test cases are necessary.
6. Tools: The required test tools must be identified including the owner for their development or acquisition and how to use them or when they are needed.
   
7. Computer time: Each testing phase's required computer time should be calculated.
8. Hardware configuration:
9. Integration: System integration plan should be in place that defines the order of integration and the functional capability of each version of the system.
   
10. Tracking procedures: There should be tracking of errors and the estimation of progress with respect to schedule, resources and completion criteria.
11. Debugging procedures: Mechanisms must be in place to track the progress of corrections and adding them to the system.
    
12. Regression testing: Regression testing is important because changes and error corrections tend to be more error-prone than the original code. The purpose of the regression testing is to determine if the change has regressed other aspects of the program.
    

The author then discusses about test completion criteria which can be two of the following:

1. Scheduled time for testing expires.
2. When all the test cases execute without detecting any errors.

The author describes how both the criteria are useless. The author provides three criteria

1. The first one is based on use of testing methodologies:
• Completion of module testing can be achieved if the test cases are derived from multi-condition coverage criterion or boundary value analysis of interface specification and all the test cases are unsuccessful.
• Completion of function testing can be achieved if the test cases are derived from cause-effect graphing, boundary value analysis and error guessing and all the test cases are unsuccessful.
2. The second criteria is to state the completion requirements in positive terms, i.e. test of a module is not complete till x errors are found or an elapsed time of y months. The author discuss the problem of estimating the number of errors in a program and provides several ways like
• Experience with previous programs
• Apply predictive models
• Use industry-wide averages
3. The third criteria is to plot the number of errors found per unit time during test phase. By examining the shape of the curve, it can be determined whether to finish or continue the testing.
   

The author again emphasizes on hiring an independent test agency to test the program.

The Art of Software Testing by Glenford Myers : Chapter 5

The author discusses about unit testing in this chapter and enlists following advantages of module testing:

• It makes combinatorics of managing testing easier.
• It makes debugging easier.
• It speeds up testing since multiple modules can be tested simultaneously.

The author then takes an example and generates test cases for it.

The author then discusses incremental testing where one module at a time is integrated with already tested modules and then tested. In non-incremental testing, all the modules are first tested and then integrated to create a program. The author then provides following observations for incremental testing:

1. Non-incremental testing requires more work.
2. Programming errors related to mismatching interfaces or incorrect assumptions among modules will be detected earlier if incremental testing is used.
3. Debugging becomes easier.
4. Incremental testing results in thorough testing.
5. Non-incremental approach uses less machine-time.
6. Non-incremental testing can be more parallelized in the earlier phase.

The author then discuss two approaches of incremental testing:

• Top-down approach: The author points out following shortcomings of this approach:
• When a new module is introduced, it is impossible to test some predefined situations.
• If the newly introduced module is at a distance from the module doing I/O operations, it is difficult to decide the set of inputs that allows to execute all tests on the newly introduced module.
• Since displayed output is from a module large distance away from the module being tested, correlation between output and behaviour of the module is difficult to establish.
• Even though it seems possible, it cannot be combined with top-down design.
• Sometimes, a module may remain untested before moving to the other module.
• Bottom-up approach: The author points out following advantages/disadvantages of this approach:
• The working program does not get created till the last module is added.

The author then provides a list of guidelines to perform the actual test:

• The set of test cases needs to be reviewed or inspected before the test is performed.
• Use automated test tools.
• Look at the side effects of a module.
• Testing of a module should be done by programmer of a calling module.
• Debugging should always be done by the programmer.
• If a high number of errors are found in a subset of the modules, such modules should be subjected to further module testing.