7print.bg = 1 принтиране + 2 плотиране + 3 подвързване + 4 чаши + 5 тениски + 6 книгоиздаване + 7 рекламни м-ли 088 682 47 47

Категории

Automated Software Testing Dustin Rashka Paul

Сумата се прибавя директно в кошницата

10.80лв

Описание

Contents

Preface xv

A ck n o ivledgm e n ts xxi

What Is Automated Testing?

1 The Birth and Evolution of Automated Testing 3

1,1 Automated Testing 3

1.2 Background on Software Testing 5

1.3 The Automated Test Life-Cycle Methodology' (ATLM) 7

1.3.1 Decision to Automate Test 10

1.3.2 Test Tool Acquisition 12

1.3.3 Automated Testing Introduction Phase 12

1.3.4 Test Planning, Design, and Development 13

1.3.5 Execution and Management of Tests 14

1,3.6 Test Program, Review and Assessment 14

1.4 ATLM’s Role in the Software Testing Universe 14

1.4.1 ATLM Relationship to System Development Life Cycle 14

1.4.2 Test Maturity Model (TMM)—Augmented by Automated Software Testing Maturity 15

1.4.3 Test A utoma tion Development 19

1.4.4 Test Effort 21

1.5 Software Testing Careers 22

2 Decision to Automate Test

2.1 Overcoming False Expectations for Automated Testing

2.1.1 Automatic Test Plan Generation

2.1.2 Test Tool Fits All

2.1.3 Immediate Test Effort Reduction

2.1.4 Immediate Schedule Reduction

2.1.5 Tool Ease of Use

2.1.6 Universal Application of Test Automation

2.1.7 One Hundred Percent Test Cover яде

2.2 Benefits of Automated Testing

2.2.1 Production of a Reliable System

2.2.2 Improvement of the Quality of the Test Effort

2.2.3 Reduction of Test Effort and Minimization of Schedule Case Study: Value of Test Automation Measurement

2.3 Acquiring Management Support

2.3.1 Test Tool Proposal t>b

3 Automated Test Tool Evaluation and Selection 67

3.1 The Organization’s Systems Engineering Environment 70

3.1.1 Third-Party Input from Management, Staff, and Customers and End Users 71

3.1.2 Tool Criteria Reflecting the Systems Engineering Environment 72

3,1.3 Level of Software Quality 73

3.1,4 Help Desk Problem Reports 74

3.1.5 Budget Constraints 74

3.1.6 Types of Tests 74

3.1.7 Long-Term Investment Considerations 75

3.1.8 Test Tool Process 75

3.1.9 Avoiding Shortcuts 75

3.2 Tools That Support the Testing Life Cycle 76

3.2.1 Business Analysis Phase Tools 79

3.2.2 Requirements Definition Phase Tools 80

3.2.3 Tools for the Analysis and Design Phase 82

3.2.4 Programming Phase Tools 83

3.2.5 Metrics Tools 85

3.2.6 Other Testing Life-Cycle Support Tools 86

3.2.7 Testing Phase Tools 86

3.3 Test Tool Research 89

3.3.1 Improvement Opportunities 89

3.4 Evaluation Domain Definition 96

3.5 Hands-On Tool Evaluation 98

3.5.1 Evaluation Report 99

3,5.2 License Agreement 101

Fart I!

Introduction of Automated Testing to a Project

4 Automated Testing Introduction Process 107

4.1 Test Process Analysis 11 ()

4.1.1 Process Review 1 ] 2

4.1.2 Goals and Objectives of Testing 1 ] 6

Case Study: Test Objectives and Strategics 119

4.1.3 Test Stra legits 120

4.2 Test Tool Consideration 133

4.2.1 Review of Project-Specific System Requirements 135

4.2.2 Application-Under-Test Overview 137

4.2.3 Review of Project Schedule 138

4.2.4 Test Tool Compatibility Check 139

4.2.5 Demonstration of the Tool to the Project Team 140

4.2.6 Test Tool Support Profile 141

4.2.7 Review of Training Requirements 143

5 Test Team Management 147

5.1 Organizational Structure of a Test Team 149

5.1.1 Stovepipe Test Team 151

5.1.2 Centralized Test Team 151

5.1.3 IV&V Test Team 153

5.1.4 Systems Methodology and Test Team 154

5.1.5 Test Tea m Su m ma ry 155

5.2 Test Program Tasks 157

5.3 Test Effort Sizing 163

5.3.1 Test Team Sizing Methods: An Overview 165

5.3.2 Development Ratio Method 165

5.3.3 Percentage Method 166

5.3.4 Test Procedure Method 167

5.3.5 Task Planning Method 168

5.3.6 Test Effort Sizing Factors 170

Test Engineer Recruiting 172

5.4.1 Test Engineer Qualities 172

5.4.2 Test Team Composition 174

5.4.3 Job Requisition 176

5.4.4 Recruiting Activities 178

5.4.5 Locating Test Engineers 178

5.4.6 Test Engineer Interviews 179

5.4.7 Distinguishing the Best Candidate 181

Roles and Responsibilities 182

Part Ilk

Test Planning and Preparation

Test Planning: Smart Application of Testing 191

6.1 Test Planning Activities 192

6.2 Test Program Scope 197

6.2.1 System Description 198

6.2.2 Critical/High-Risk Functions 199

6.2.3 Test Goals, Objectives, and Strategies 199

6.2.4 Test Tools 199

6.2.5 Test Program Parameters 200

6.2.6 Verification Methods 202

6.2.7 Test Requirements Definition 203

6.3 Test Requirements Management 205

6.3.1 Requirements Management Tools 206

6.3.2 Assessing the Test Requirements Risk 208

6.3.3 Prioritization of Tests 208

6.3.4 Requirements Traceability Matrix 209

6.4 Test Program Events, Activities, and Documentation 211

6.4.1 Events 211

6.4.2 Activities 212

6.4,3 Documentation 213

6.5 The Test Environment 214

6.5.1 Test Environment Preparations 214

6.5.2 Test Environment Integration and Setup 216

6.6 The Test Plan 217

6.6.1 Test Completion/Acceptance Criteria 220

6.6.2 Sample Test Plan 220

7 Test Analysis and Design 223

7.1 Test Requirements Analysis 225

7.1.1 Development-Level Test A nalysis

(Structural Approach) 226

7.1.2 System-Level Test Analysis (Behavioral Approach) 228

7.2 Test Program Design 233

7.2.1 Test Fragra m Design Models 233

7.2.2 White-Box Techniques (Development-Level Tests) 237

7.2.3 Black-Box Techniques (System-Level Tests) 244

7.2.4 Test Design Documentation 255

7.3 Test Procedure Design 256

7.3.1 Test Procedure Definition 257

7.3.2 Automated Versus Manual Test Analysis 262

7.3.3 Automated Test Design Standards 266

Case Study: Naming Conventions 270

7.3 A Manual Test Design Guidelines 272

7.3.5 Detailed Test Design 274

7.3.6 Test Data Requirements 277

8 Test Development 285

8.1 Test Development Architecture 288

8.1.1 Technical Environment 288

8.1.2 Environment Readiness Checks 291

8.1.3 Automation Framework Reuse Analysis 291

8.1.4 Test Procedure Development/Execution Schedule 292

8.1.5 Modularity-Relationship Analysis 295

8.1.6 Explanation of the Sample Modularity-

Relationship Matrix 299

8.1.7 Calibration of the Test Tool 302

8.1.8 Compatibility Work-Around Solutions 302

Case Study: Incompatibility Work-Around Solution 303

8.1.9 Manual Execution of Test Procedures 304

8.1.10 Test Procedure Inspections—Peer Reviews 304

8.1.11 Test Procedure Con figuration Management 305

8.2 Test Development Guidelines « 306

8.2.1 Design-to-Development Transition 308

8.2.2 Reusable Test Procedures 310

Case Study : Navigation Using Tabs or Mouse Clicks 313

Case Study: Testing Bitmaps Using a Capturc/Playback Tool 314

8.2.3 Maintainable Test Procedures 317

Case Study: Automating Documentation 320

Case Study: Automated Random Testing 332

8.2.4 Other Guidelines 334

8,3 Automation Infrastructure 336

8.3.1 Table-Driven Test Automation 337

8.3.2 PC Environment Automated Setup Script 339

8.3.3 Automated Recording Options 340

8.3.4 Login Function 341

8.3.5 Exit Function 341

8.3.6 Navigation 341

8.3.7 Verifying GUI Standards 342

8.3.8 Smoke Test 342

Case Study: Smoke Test Application 343

8.3.9 Error-Logging Routine 343

8.3.10 Help Function Verification Script 343

8.3.11 Timed Message Boxes Function 344

8.3.12 Advanced Math Functions 344

ijA IV

Test Execution and Review

§ Test Execution 349

9.1 Executing and Evaluating Test Phases 351

9.1.1 Unit Test Execution and Evaluation 351

9.1.2 Integration Test Execution and Evaluation 354

9.1.3 System Test Execution and Evaluation 356

9.1.4 Test Results Analysis of Regression Tests 358

9.1.5 User Acceptance Test Execution and Evaluation 359

9.2 Defect Tracking and New Build Process 360

9.2.1 Defect Life-Cycle Model 366

9.3 Test Program Status Tracking 366

9.3.1 Earned Value Management System 367

Case Study: System Test Status Tracking 367

9.3.2 Test Metrics Collection and Analysis 370

1§ Test Program Review and Assessment 379

10.1 Test Program Lessons Learned—Corrective Actions

and Improvement Activity 381

10.2 Test Program Return on Investment 392

Case Study: Test Program Return on Investment 392

Case Study: Quantify Tool Return on Investment 397

"art V Appendixes

A. How to Test Requirements 405

A. 1 Requirements Testing Approach 405

Abstract 405

The Quality Gateway 406

Make the Rcquhemcih Mwi.niauit 406

Qunurifi'ilL Rujttl, тапа 406

II,,uj„.,nCf.„l,d iCi,..» n:: 407

Keeping Track 407

Coherency and Consistency 408

Completeness 409

Relevance 411

Requirement or Solution ? 412

Stakeholder Value 413

Traceability 413

Order in a Disorderly World 414

Conclusions 415

References 416

B Tools That Support the Automated Testing Life Cycle 417

B. 1 Introduction 417

B.2 Business Analysis Phase 421

B.2.1 Business Modeling Tools 421

B.2.2 Configuration Management Tools 424

B.2.3 Defect Tracking Took 426

B.2.4 Technical Review Management 432

B.2.5 Documentation Generators 433

B.3 Requirements Definition Phase 434

B.3.1 Requirements Management Tools 434

B.3.2 Requirements Verifiers u 436

B.3.3 Use Case Generators 436

B.4 Analysis and Design Phase 437

B.4.1 Visual Modeling Tools 437

B.4.2 Structure Charts, Flowcharts, and Sequence Diagrams 440

B.4.3 Test Procedure Generators 441

B.5 Programming Phase 442

B.5.1 Syntax Checkers/Delmggers 442

B.5,2 Memory Leak and Runtime Error Detection Tools 442

B.5.3 Code Checkers 444

B.5.4 Static and Dynamic Analyzers 445

B.5,5 Unit Test Tools 449

B.6 Metrics Tools 450

B.6.1 Code (Test) Coverage Analyzers and

Code Instrumentors 450

B.6.2 Usability Measurement Tools 456

B.7 Testing Support Tools 457

B.7.1 Test Data Gen cm tors 457

B.7.2 File Comparison Tools 458

B.7.3 Simulation Tools 459

B.8 Testing Phase 460

B.8.1 Test Management Tools 460

B.8.2 Network Testing Tools 460

B.8.3 GUI Application Testing Tools 462

B.8.4 Load/Performancc Testing Tools 465

B.8.5 Web Testing Tools 470

B.8.6 Tear 2000 Testing Tools 471

B. 9 Other Test Tool Vendors 472

C Test Engineer Development 475

C. 1 Technical Skills Stage 478

C.2 Test Process Stage 480

C.3 . Team Effort Stage 483

C.4 Technical Stewardship Stage 484

C.5 Test/Project Management Stage 485

C. 6 Business/Product Management Stage 487

D Sample Test Plan 489

D. 1 Introduction 493

D.1.1 Purpose 493

D.1.2 Background 493

D.1,3 System Overview 494

D.1,4 Applicable Documents 496

D.1,5 Master Schedule 498

D.2 Roles and Responsibilities 499

D.2.1 Project Organization 499

D.2.2 Project Roles and Responsibilities 500

D.2.3 Test Task Structure 503

D.2.4 Test Team Resources 509

D.3 Test Program 509

D.3.1 Scope 509

D.3.2 Test Approach 512

D.3,3 Test Strategics 515

D.3.4 Automated Tools 51.8

D.3.5 Qualification Methods 519

D.3.6 Test Requirements 519

D.3.7 Test Design o20

D.3.8 Test Development 524

D.4 Test Environment 526

D.4.1 Test Environment Configuration 526

D.4.2 Test Data 527

D.5 Test Execution 529

D.5.1 Test Program Reporting 529

D.5.2 Test Program Metrics 529

D.5.3 Defect Tracking 530

D.5.4 Configuration Management 532

D. 6 Detailed Test Schedule 532

APPENDIXES 534

D.A Test Procedure Development Guidelines 534

D.B Test Verification Summary and Matrix 536

D.C Test Procedures and Test Scripts 538

Best Practices 539

E. 1 Documented Process 539

E.2 Managing Expectations 541

E.3 Pilot Project 541

E.4 Test Tool Compatibility Checks 541

E.5 Test Tool Upgrades sj 542

E.6 Baselined System Setup and Configuration 543

E.7 Software Installations in the Test Environment Baseline 543

Е.8 Overall Test Program Objectives 543

E.9 Keep Automation Simple 544

E.10 Test Procedure Design and Development Standards 544

Е.11 Automated Versus Manual Test Analysis 545

Е.12 Reuse Analysis 545

Е.13 Test Team Communication with Other Teams 545

Е.14 Schedule Compatibility' 546

Е.15 Customer Involvement 546

Е.16 Defect Documentation and Reporting 547

Е,17 Automated Test Advocates and Experts 547

Е,18 Test Team Assignments 547

Е.19 User Group Participation 548

Е.20 Test Tool Improvement Suggestions 548

Е.21 Become a Beta Testing Site 548

Е.22 Specialty' Topic Experts 548

Index 551