For an allegory of quality assurance (“QA”), I’d like to turn to one of the more fantastic episodes in the history of my home of Los Angeles: Howard Hughes failed flying behemoth, the H-4 Hercules (a wooden craft more commonly known as the “Spruce Goose”):
Howard Hughes signed an independent contract with the War Department to produce the aircraft during the Second World War. Although he had originally been partnered with a ship-builder, Howard Kaiser, the two parted ways because of production delays Kaiser’s annoyance at Hughes insistence on “perfection.” Hughes continued to work on the project until well after the war ended.
In 1947, Hughes was called to testify before the Senate War Investigating Committee to justify his use of government funds. During a Senate hearing on August 6 (the first of a series of appearances), Howard Hughes said:
The Hercules was a monumental undertaking. It is the largest aircraft ever built. It is over five stories tall with a wingspan longer than a football field. That’s more than a city block. Now, I put the sweat of my life into this thing. I have my reputation all rolled up in it and I have stated several times that if it’s a failure, I’ll probably leave this country and never come back. And I mean it.
During a break in the Senate hearings, Hughes returned to California to run taxi tests on the H-4. On November 2, the taxi tests began with Hughes at the controls. His crew included Dave Grant as co-pilot, two flight engineers, Don Smith and Joe Petrali, 16 mechanics, and two other flight crew. After picking up speed on the channel facing Cabrillo Beach, the Hercules lifted off, remaining airborne at 70 ft (21 m) off the water and a speed of 135 miles per hour (217 km/h) for around a mile (1.6 km).
Having proven to his detractors that Hughes’ masterpiece was flight-worthy, thus vindicating the use of government funds, the “Spruce Goose” never flew again. Its lifting capacity and ceiling were never tested. A full-time crew of 300 workers, all sworn to secrecy, maintained the aircraft in flying condition in a climate-controlled hangar. The crew was reduced to 50 workers in 1962, and then disbanded after Hughes’ death in 1976.
Howard Hughes flight of the “Spruce Goose” is a fascinating story, when examined from the perspective of performance. In terms of performance theory, the single flying run of the H-4 can be considered a “nonce” act, an event made necessary by its circumstances which, by being enacted, also changes them. To these ends, Hughes’ Hercules’ one flight was necessary in order to prove that the machine could satisfy its purposive duty (flight), in order to substantiate the illocutionary value of the monies applied to it (given by the US government, ostensibly to produce something which could fly); having satisfied this intention, the machine no longer needed to operate, for it had already been proven capable of doing so. Circumstantially, the demand of Senate War Investigating Committee to qualify the Spruce Goose as a worthy use of funds had the perlocutionary effect shuttering the project and the apparatus. Many historians and commentators have claimed that the plane’s flight was the effect of ground effect (whereby objects experience additional lift and depreciated drag when near and parallel to a larger, flat surface), and it is likely that Hughes though this himself. To ensure that this disinterpellation of the Spruce Goose from the category of flying things could not be questioned, the machine was put into suspended animation. In its climate-controlled chamber, the H-4 remained as it was in that brief instance, aloft: physically arrested at and dutifully maintained in the physical state and institutional gravitas it exhibited in November 1947. With this in mind, one can argue that the flight of the Spruce Goose was a cover for the shame and denial of its stakeholders’ failed investiture: a complete and perfect non-exposure, nothing admitted and nothing denied.
The theorist within me salivates over this episode as a metaphor for the shame of societal expectations. Putting this interest aside for a moment, I would like to indulge the event of the “flight of the Spruce Goose” as a possible doppleganger for the Scrum development process, and the function of QA within it.
Scrum is an iterative software development project management methodology. It focuses on flexible and holistic strategies, where team members work as a unit to reach a common goal. Scrum recognizes that customers change their minds, and that challenges often emerge unplanned; Scrum therefore adopts an approach whereby projects are not sought to be understood, and focuses instead on maximizing team ability to deliver on ever-evolving requirements. In Scrum, QA is an integral component: working software is the prize outcome; this means that QA must be integrated into the development process from the earliest stages.
Although MITx does not follow Scrum project management strategies explicitly, 4.605x was developed following some of its principles: the team works together to make something that meets faculty expectations in a user friendly manner, and that is our goal first, foremost, and always. Intuitively, we on the course team constantly checked to ensure the quality of our offering by demoing our content with MITx staff and Professor Jarzombek, and gathering thoughts from friends and peers.
Now that our course has reached initial completion, the team is in the process of reaching out to a body of about two dozen pre-selected beta testers. These beta-testers will be granted early access to the course, and after some initial exploration will report back on basic measures of program quality and usability. The purpose of this limited exploration is to ensure that our content is indeed in good working order. Based on the assumption of the release of one Lecture’s worth of material, the questions we have drafted are as follows:
- Are the instructions provided easy to understand? Where did you have trouble accessing content or navigating and why?
- Are all of the links in good working order? Are any broken? Which? 3. Is the video clear? Do you have any trouble watching it?
- Did the learning sequence have a good flow? Did the organization make sense to you?
- Did you review our tutorial material? Was it clear and comprehensive?
- Were there too many questions posed?
- What other recommendations or suggestions do you have to improve this course?
I bring up the incident of the Spruce Goose and Senate hearings for the reason that this pattern–interrogation and limited exploration of abilities–is shared between that historic series of events and our QA process, especially with its use of beta testers. It is entirely likely that our testers will enter into our system, make it only so far, and then start to encounter errors which start to question the overall integrity of the course and its orientation towards our desired outcome. This I see as an online courseware version of only flying because of the virtues of ground effect: in this situation, it would remain unclear if the course is really ready to launch.
The pressure on us is high, because all of us want our course to be something we can be proud of. It would certainly be a shame were we to have people give the course a test drive, for them to find things barely passable, and then put to the project in cold storage (lest it be deemed a failure).
Considering we already have a community of about 15,000 potential students, and content release is scheduled to happen automatically, the likelihood of such shelving is slim; however, the cautionary potential of the H-4 remains: we have been wary not to place too high of predetermined expectations upon ourselves, and by integrating this initial (and quite possibly flawed!) launch into our design process, we ensure that failure to launch can be catalyzed as a chance to improve the offering we will eventually release out unto the world.
Indulging my metaphor and allusions, here: 4.605x will fly, when expected to, even if we don’t quite make it off the ground just yet.
— Samuel Ray Jacobson, MITx Fellow