What is Alpha Testing ?
Alpha testing is a type of acceptance testing; performed to identify all possible issues/bugs before releasing the product to everyday users or public. The focus of this testing is to simulate real users by using blackbox and whitebox techniques. The aim is to carry out the tasks that a typical user might perform. Alpha testing is carried out in a lab environment and usually the testers are internal employees of the organization. To put it as simple as possible, this kind of testing is called alpha only because it is done early on, near the end of the development of the software, and before Beta Testing.
What is Beta Testing?
Beta Testing of a product is performed by "real users" of the software application in a "real environment" and can be considered as a form of external User Acceptance Testing.
Beta version of the software is released to a limited number of end-users of the product to obtain feedback on the product quality. Beta testing reduces product failure risks and provides increased quality of the product through customer validation.
It is the final test before shipping a product to the customers. Direct feedback from customers is a major advantage of Beta Testing. This testing helps to tests the product in real time environment.
Alpha Testing Vs Beta testing:
Following are the differences of Alpha and Beta Testing:
|Alpha testing performed by Testers who are usually internal employees of the organization||Beta testing is performed by Clients or End Users who are not employees of the organization|
|Alpha Testing performed at developer's site||Beta testing is performed at client location or end user of the product|
|Reliability and Security Testing are not performed in-depth Alpha Testing||Reliability, Security, Robustness are checked during Beta Testing|
|Alpha testing involves both the white box and black box techniques||Beta Testing typically uses Black Box Testing|
|Alpha testing requires lab environment or testing environment||Beta testing doesn't require any lab environment or testing environment. Software is made available to the public and is said to be real time environment|
|Long execution cycle may be required for Alpha testing||Only few weeks of execution are required for Beta testing|
|Critical issues or fixes can be addressed by developers immediately in Alpha testing||Most of the issues or feedback is collected from Beta testing will be implemented in future versions of the product|
|Alpha testing is to ensure the quality of the product before moving to Beta testing||Beta testing also concentrates on quality of the product, but gathers users input on the product and ensures that the product is ready for real time users.|
Types of Beta Testing
There are different types of Beta tests, and they are as follows:
Traditional Beta testing: Product is distributed to the target market, and related data is gathered in all aspects. This data can be used for Product improvement.
Public Beta Testing: Product is publicly released to the outside world via online channels and data can be gathered from anyone. Based on feedback, product improvements can be done. For example, Microsoft conducted the largest of all Beta Tests for its OS -- Windows 8 before officially releasing it.
Technical Beta Testing: Product is released to the internal group of an organization and gather feedback/data from the employees of the organization.
Focused Beta: Product is released to the market for gathering feedback on specific features of the program. For example, important functionality of the software.
Post release Beta: Product is released to the market and data is gathered to make improvements for the future release of the product.
Entry and Exit criteria
Entry Criteria for Alpha testing:
- Software requirements document or Business requirements specification
- Test Cases for all the requirements
- Testing Team with good knowledge about the software application
- Test Lab environment setup
- QA Build ready for execution
- Test Management tool for uploading test cases and logging defects
- Traceability Matrix to ensure that each design requirement has alteast one Test Case that verifies it
Exit Criteria for Alpha testing
- All the test cases have been executed and passed.
- All severity issues need to be fixed and closed
- Delivery of Test summary report
- Make sure that no more additional features can be included
- Sign off on Alpha testing
Entrance criteria for Beta Testing:
- Sign off document on Alpha testing
- Beta version of the software should be ready
- Environment ready to release the software application to the public
- Tool to capture real time faults
Exit Criteria for Beta Testing:
- All major and minor issues are closed
- Feedback report should be prepared from public
- Delivery of Beta test summary report
Phases of Testing
Alpha and Beta tests are typically carried for "off-the shelf" software or product oriented companies. The Phases of Testing for a product company typically varies from a service oriented organization. Following is the testing phase adopted by product firms
Pre-Alpha:- Software is a prototype. UI is complete. But not all features are completed. At this stage software is not published.
Alpha: Software is near its development and is internally tested for bugs/issues
Beta: Software is stable and is released to limited user base. The goal is to get customer feedback on the product and make changes in software accordingly
Release Candidate (RC): Based on the feedback of Beta Test, you make changes to the software and want to test out the bug fixes. At this stage, you do not want to make radical changes in functionality but just check for bugs. RC is also put out to the public
Release: All works, software is released to public.
Note: Above is a standard definition of the Testing stages but in order to garner marketing buzz, companies combine stages like "pre-alpha beta", "pre-beta" etc.
Advantages & Disadvantages
Advantages Beta Testing
- Reduces product failure risk via customer validation.
- Beta Testing allows a company to test post-launch infrastructure.
- Improves product quality via customer feedback
- Cost effective compared to similar data gathering methods
- Creates goodwill with customers and increases customer satisfaction
Disadvantages Beta Testing
- Test Management is an issue. As compared to other testing types which are usually executed inside a company in a controlled environment, beta testing is executed out in the real world where you seldom have control.
- Finding the right beta users and maintaining their participation could be a challenge
Advantages of Alpha Testing:
- Provides better view about the reliability of the software at an early stage
- Helps simulate real time user behavior and environment.
- Detect many showstopper or serious errors
- Ability to provide early detection of errors with respect to design and functionality
Disadvantages of Alpha Testing:
- In depth functionality cannot be tested as software is still under development stage Sometimes developers and testers are dissatisfied with the results of alpha testing
Myths of Alpha and Beta Testing:
Myth: Alpha and Beta testing are carried out too late in the testing life cycle to merit a benefit
Truth: Alpha and Beta testing help highlight crucial issues with the software application and provide user feedback.
Myth: Same type of testing is done in Alpha and Beta Testing and they are not different scenarios
Truth: Alpha Testing is done in a lab environment, whereas Beta testing is done by real users reporting real problems based on firsthand experience. Both scenarios are remarkably different.
Myth: Beta type of testing is difficult and time consuming
Truth: Beta testing takes time/experience to obtain the most value. But the data/feedback it provides is invaluable.
Myth: Beta testing generates little or no useful data
Truth: A successful Beta Test can generate tons of valuable information which is otherwise difficult to procure in a lab environment.
No matter how many tests you perform, how many bugs you kill, your software is useless if your users do not like it. Beta testing (second Greek alphabet)helps provide authentic feedback of your software from real users.
Alpha testing (first Greek alphabet) helps simulate realtime user environment before the software is sent for Beta Testing and helps shape a stable software candidate eligible for Beta Tests.
Alpha and Beta Testing are indispensable in your testing life cycle.
Further information: Army Beta
The Army Alpha is a group-administered test developed by Robert Yerkes and six others in order to evaluate the many U.S. military recruits during World War I. It was first introduced in 1917 due to a demand for a systematic method of evaluating the intellectual and emotional functioning of soldiers. The test measured "verbal ability, numerical ability, ability to follow directions, and knowledge of information". Scores on the Army Alpha were used to determine a soldier's capability of serving, his job classification, and his potential for a leadership position. Soldiers who were illiterate or foreign speaking would take the Army Beta, the nonverbal equivalent of the exam.
Army Beta The Development of the beta test and of the performance test for the examination of the foreign speaking and illiterate presented special problems. The use of demonstration charts and mime to convey the instructions to the persons being examined proved successful. The new type of the test in the beta, using geometrical designs, mutilated pictures, etc., required different principles in its construction. The individual performance tests also involved additional and peculiar standards of construction and evaluation.
The important purpose of these supplementary tests was, of course, to give to those handicapped by language difficulties a real opportunity to show their ability. In addition, two definite aims were planned in the use of all forms of testing: first, to point out the feeble-minded and those incapable of military service because of mental deficiency; and second, to find those of unusual or special ability. The arrangement of each test, in both group and individual examinations, was therefore checked against the sources of men in institutions for the feeble-minded. If no score had meant low mentality, the first task would have been solved; but it had been shown that literacy was an important factor in the alpha test. The beta test practically eliminated this factor and was thus a step further in selecting those of low intelligence. To prove conclusively that a man was weak-minded and not merely indifferent or malingering, the performance test was added.
The individual examinations as finally used in the U.S. Army were, therefore, primarily checks on the group examinations. No person was reported as feeble-minded until a detailed individual psychological examination had been made. Many cases of mental disorder were discovered and referred to the psychiatrists for examination. Disciplinary cases referred to the psychologists were always given individual examinations, as were referred cases of men having difficulty with drill or those who failed to improve in the YMCA schools and elsewhere. Both the Army Alpha and Army Beta tests were discontinued after World War I.
Relationship of Scores and Errors
In any psychological aptitude test, the person scoring the test has to take into consideration any error that the examinee will possibly make while taking the test. C. R. Atwell did a small study on the relationship of scores and errors based on the results of administration of the "Army Alpha".
C. R. Atwell (1937, p. 451) wrote, "The number of errors made by a subject on a test should be indicative of his approach to the test, whether he works in a hurriedly and rashly or slowly and cautiously. Considered alone, however, the number of errors is a relatively meaningless figure, since more errors would be expected with lower scores. If for a given score wide deviations occur in the number of errors, the error score of a subject should be of value in giving additional information about him".
Yerkes outlined seven purposes for administering the "Army Alpha" test:
- "Classify soldiers according to their mental ability, thus supplementing personnel records of occupational qualifications and assisting with assignment in the Army
- Supply a mental rating for each soldier which shall assist personnel officers in building organizations of equal or of appropriate strength
- Assist in regimental, company and medical officers by careful examination and report on men who are not responding satisfactorily to training, or are otherwise troublesome
- Assist officers of development battalion with classification, grading, training, and ultimate assignment of men
- Assist in discovering men of superior mental ability who should be selected for officers' training camps, for promotion, or for assignment to special tasks
- Assist in discovering and properly placing men of marked special skill, as for example, observers or scouts for intelligence service
- Assist in discovering men who are mentally inferior and who in accordance with degree of defectiveness should be recommended for discharge, development battalions, labor organizations or regular military training"
Methods and results
The administration of the "Army Alpha" was carried out in the following manner:
- The obviously illiterate were segregated from the literate
- Administration of the "Army Alpha" (for literates). The time of the test was 40 to 50 minutes and there were 100 to 200 men in a group.
- Administration of the "Army Beta" (for illiterates and men who failed the "Army Alpha"). The time of the test was 50 to 60 minutes and there were up to 60 men in a group.
- Men who failed the "Army Beta" were subject to individual examinations. Literates who failed the "Army Alpha" and "Army Beta" are given the Stanford-Binet and/or the Point-Scale, depending on the subject. Illiterates who failed the "Army Beta" were given a Performance Test
On November 11, 1918, the psychological personnel consisted of about 120 officers and 350 enlisted men. Over five hundred additional clerks were used in the examining service in the 35 different camps in which psychological examining had been established. The army intelligence examination had been given to 1,726,966 men; of these 41,000 were officers. Approximately 83,000 individual examinations had been given. Over 7,800 men had been recommended for immediate discharge; 10,014 had been recommended for labor battalions or other service organizations; 9,487 had been recommended for development battalions for further observation and preliminary training. Nearly 30 percent of the 1,556,011 men for whom statistics are available were found to be unable to "read and understand newspapers and write letters home", and were given a special examination prepared for illiterates, the Army Beta.
The methods originally prepared for use in the Army were subjected to repeated revisions, in the light of results, for increase in reliability and military value. The procedure finally adopted and used throughout the Army consists of two chief types of examination: the group examination and the individual examination. The former was necessitated by the demand for speed of examination and report, the latter by the desire for reliability and fairness to the individual.
Men were examined in groups as large as five hundred. Every man was supplied with a pencil and an examination blank. He then, under military discipline, follows directions to the best of his ability. The examination required approximately fifty minutes. It demands almost no writing since responses were indicated by underscoring, crossing out, or checking. The examination papers were quickly scored by means of stencils, and mental ratings recorded for prompt report. To avoid, within reasonable limits, the risk of coaching, several duplicate forms of the examination had been made available. Each test of examination alpha consisted of a number of parts arranged in order of difficulty low to high. It was therefore possible for low-grade subjects to make a start on each test, and, at the same time, practically impossible for highly intelligent subjects to complete the tests within the time allowed. The test were varied in character and undoubtedly sample the most important types of intellectual process.
The general procedure of examining, which was developed to meet military requirements, is briefly describable as follows: A group of draftees, the size of which is determined by the seating capacity of examining room (it varied from one hundred to five hundred men) is reported to the psychological examining building for mental testing. The first essential step is the segregation of illiterates. This is accomplished by having all men who cannot read and write their own letters and those who have not proceeded beyond the fifth grade in school step out of the original group. The remaining men are sent to the alpha room. Naturally, among them there are likely to be several who will subsequently have to take the Army Beta examination. The illiterates are sent directly to the Army Beta room.
Men who fail in the alpha were sent to the Army Beta in order that injustice by reason of relative unfamiliarity with English may be avoided. Men who fail in Army Beta are referred for individual examination by means of what may appear to be the most suitable and altogether appropriate procedure among the varied methods available. This reference for careful individual examination is yet another attempt to avoid injustice either by reason of linguistic handicap or accidents incident to group examining.
The "Army Alpha" is divided into 8 tests. There are also five forms of the test—forms 5, 6, 7, 8, 9 and the questions asked are ordered in ascending difficulty. Army recruits were asked to answer the questions as quickly as possible. They tested arithmetic, "common sense", vocabulary, word order, number patterns, analogies, and sentence completion.
The Army Beta yield numerical scores or intelligence scores which for practical military purposes are translated into letter grades.
The several intelligence letter grades used in the Army, with their score-equivalents and appropriate definitions are presented here:
|Grade||Intelligence||Score (alpha)||Score (beta)||% of total||Description|
|A||Very superior||135-212||100-118||4% to 5%||Men of marked intellectuality. They are of high officer type when they are also endowed with leadership and other necessary qualities.|
|B||Superior||105-134||90-99||8% to 10%||Less exceptional than "A". Many men of the commissioned officer type and a large amount of non-commissioned officer material.|
|C+||High average||75-104||80-89||15% to 18%||A large amount of non-commissioned officer material with occasionally a man whose leadership and power to command fit him for commissioned rank.|
|C||Average||45-74||65-79||About 25%||Excellent private type with a certain amount of fair non-commissioned officer material.|
|C-||Low average||25-44||45-64||About 20%||Below average intelligence, but usually good privates and satisfactory in work of a routine nature.|
|D||Inferior||15-24||20-44||About 15%||Likely to be fair soldiers, but usually slow in learning and rarely go above the rank of private.|
Short on initiative and so need more than the usual amount of supervision. Many are illiterate or foreign.
|D-||Very inferior||0-14||0-19||Very inferior in intelligence but considered fit for regular service.|
|E||Very inferior||0-14||0-19||Men whose mental inferiority justifies their recommendation for development battalion, special service organization, rejection, or discharge.|
The majority of "D-" and "E" men are below ten years in "mental age".
The "Army Alpha's" origin was based on intelligence tests created by Alfred Binet. As World War I began to unfold, some men realized that the proper use of manpower, and more particularly of mind or brain power, would assure ultimate victory. The "Army Alpha" was created by a group of psychologists that consisted of: Robert Yerkes, W. V. Bingham, Henry H. Goddard, T. H. Haines, Lewis Terman, G. M. Whipple, F. L. Wells. Each of them brought to the committee a large amount of material that was sifted to produce the group test and individual examining material "Examiners Guide". These seven experts in practical mental measurement were organized in summer 1917 and called together to prepare or select suitable methods. They worked almost continually for a month, devising, selecting, and adapting methods. Another month was spent thoroughly testing the methods in military stations so that their value might be definitely established before they should be recommended to the Medical Department of the Army. The results were gratifying and the methods were therefore recommended to the Surgeon General of the Army in August 1917, and promptly accepted for official trial. The official medical inspector formulated these statements and recommendations: "The purposes of psychological testing are (a) to aid in segregating the mentally incompetent, (b) to classify men according to their mental capacity, (c) to assist in selecting competent men for responsible positions."
The original purposes of the committee in preparing methods for intelligence testing were less important than the uses made of the results. The committee intended as stated above to prepare an examination that would indicate the drafted men who were too low-grade mentally to make satisfactory privates in the Army; it was desired also to indicate, if possible, those who were mentally unstable or who might prove incorrigible so far as army discipline was concerned. Also, the committee hoped to be able to pick out exceptional types of men who could be used for special tasks that demanded a high degree of intelligence. In interesting contrast with these original purposes of mental examining, stand the results actually achieved.
- Assigning an intelligence rating to every soldier on the basis of systematic examination.
- Designating and selecting men whose superior intelligence indicates the desirability of advancement or special assignment.
- Promptly selecting and recommending for development battalions of men who are so inferior intellectually as to be unsuited for regular military training.
- Providing measurements of mental ability which enable assigning officers to build organizations of uniform mental strength or in accordance with definite specifications concerning intelligence requirements.
- Selecting men for various types of military duty or for special assignment, as for example, to military training schools, colleges, or technical schools.
- Providing data for forming special training groups within the regiment or battery in order that each man may receive instruction suited to his ability to learn.
- Quickly finding and recommending for elimination men whose intelligence is so inferior that they cannot be used to advantage in any line of military service.
"Examination A" was the pilot version of the "Army Alpha". 140,000 prospects were administered "Examination A" in the fall of 1917. There were actually multiple different pilot versions created. The different pilot versions include; Examination A, B, C, D, and E. "Examination A" is the most talked about because it was the first pilot administered and virtually the beginning of the testing of the "Army Alpha". Robert Yerkes and his members asked the question, "How should the results of widely distributed testing of this sort be interpreted?" As a result of these different forms of pilot tests, Robert Yerkes and the other members found decided that, "in this connection it may be emphasized again that the group examination used in the Army was interpreted entirely in terms of military need". They made necessary changes to the test to measure an individual's abilities to place them in the right field of the Army.
This is more about the history of how the test was constructed. "The test to be devised for army use the committee believed should:
- First, be adaptable for group use for the examining of large numbers rapidly.
- Second, it should be a high degree of validity as a measure of intelligence.
- Third, the range of intelligence measured by the tests should be wide; that is, the test should be made difficult enough to measure the higher levels of intelligence and at the same time be an adequate measure of the extremely low levels that would probably be found in the Army.
- Fourth, as far as possible, it should be arranged for objectivity of scoring and the elimination of personal judgment concerning correct answers; thus the results of scoring in one camp would be strictly comparable with those obtained in another.
- Fifth, the test should be so arranged that the scoring could be done rapidly and with the least chance of error. Also, this arrangement should be so simple that relatively inexpert assistance could be used in scoring the large numbers of papers.
- Sixth, there must be either different forms or alternative tests of equal difficulty to prevent coaching.
- Seventh, it was necessary also to obtain clues which would enable examiners to detect malingering in connection with the examination.
- Eighth, cheating must also be avoided.
- Ninth, the test must be made as completely independent of schooling and educational advantages as possible.
- Tenth, the arrangement should be such as to allow a minimum of writing in recording answers.
- Eleventh, the tests must consist of material which would arouse interest in the subjects.
- Twelfth, the different tests used should be arranged to yield an accurate measure of intelligence in a reasonably short time." With these criteria in mind the committee set to work on the materials available to produce what is now known as the army mental test.
Originally, there were 13 tests. These were rated by the psychologists present on the basis of their validity as measures of intelligence. These were given to selected groups and the results compared with the criteria laid down. As a result, certain tests were eliminated because they failed to meet the requirements. For each test a series of sample items correctly answered was given. Psychologists spent over two months in the study of results and in the revision methods. From this study of the results, the nonverbal group examination beta was prepared for examiners to make a rapid survey of the 30 percent who either could not read English or read it too slowly. The Stanford-Binet and the Point Scale were adapted for army use at this time and the individual examination for foreigners and illiterates was prepared.
The Army Alpha: First Nebraska Edition
The "Army Alpha": First Nebraskaa Edition was a revision that used four earlier forms of the "Army Alpha". It was revised in 1937. The most diagnostic items were selected and items referring unnecessarily to military affairs or depending upon out-of-date information were eliminated. The items are arranged in order of difficulty as empirically determined."
"The make-up and scoring follow the plan of the original Alpha. The directions are given orally for each item in the directions test, also separately timed, and oral directions precede each of the subtests. This has often seemed to be one of the inconvenient features of the Alpha, especially when used with college and adult subjects".
"Norms were apparently not recalculated from an administration of the present revision, as "the average level of difficulty for each test is the same as that for the original forms of the Army Alpha, and so the customary norms should, therefore, apply". Norms are given in terms of percentiles for the adult population. An estimated mental age and IQ may also be obtained from the same table. The norms for IQ are derived from a comparison of the percentile norms with those offered by Terman and Merrill for the distribution of IQ's in the American white population".
"The manual contains no data on reliability or validity, presumably as high or higher than the original. The test should serve as a profitable research device or as a preferred substitute for the earlier Alphas when given in the original manner."
Army Group Examination Alpha: Schrammel-Brannin Revision
The Army Group Examination Alpha: Schrammel-Brannin Revision was published in 1936. "Three of the original five forms of the five forms of the army test, with modifications and the introduction of new items. The items are equal from form to form. The eight subtests of the original Alpha have been retained. The scoring is easier, the responses being digits or a plus or minus placed in parentheses to the left of each item. Each item is not timed individually. Oral Directions were eliminated, and the test is self-administered. The examiner records the time for the group at five-minute intervals, allowing a later return to earlier items not completed if time allows. The items are presumably in order of difficulty. Scoring was done by cardboard keys supplied with the test."
"Grade, age, and a few occupational norms are supplied. The grade norms extend from the fourth grade through the elementary, high school, and college level. The number of cases for each grade averages around 700, with over 2,000 college freshmen and about 300 for each of the other college years. Age norms extend from 8 to 25 and above. There is progressive increase in the median score up to age 17, after which the increase is irregular. The norms would seem to be best from ages 9 to 17, both on this point and on the number of cases. The tentative norms are based almost entirely on school and college groups, a fact evidently to be taken into consideration if the subjects to be tested are not students." The occupational norms were tentatively for Civilian Conservation Corps camps, job applicants to GE, police officers, student nurses, and public school teachers.
"The mimeographed manual gives no data on reliability, which perhaps would be high, nor on validity, which would probably also be fairly good for the measurement of abstract intelligence or scholaptitude. The elimination of oral directions, the existence of three forms, and the modification for ease of scoring are appealing features of the test."
One of the most obvious criticisms of the "Army Alpha" is that testing has evolved so much since World War I that the information purposed in the demographics of the intelligence test is out of date. Although, a lot of the formats, even some of the questions, used from the "Army Alpha" are still around today.
There were even criticisms when the test was originally accepted to be used. Yerkes and Yoakum (1920) wrote, "When psychological examining was originally accepted by the Medical Department for Official trial, there was extreme and widely prevalent skepticism even among psychologists themselves concerning the reliability of the measurements of intelligence which could be secured and still more concerning their practical value to the Army".
Another problem that arose during its creation and administration, was that psychologist examiners had a difficult time basing a soldiers future with the Army solely on a test. Yerkes and Yoakum (1920) state, "It was repeatedly stated and emphasized by psychological examiners that a man's value to the service should not be judged by his intelligence alone, but that instead temperamental characteristics, reliability, ability to lead and to "carry on" under varied conditions should be taken into account. Even after the feasibility of securing a fairly reliable measure of ever soldier's intelligence or mental alertness had been demonstrated, it remained uncertain whether these measurements would correlate positively with military value to a sufficient degree to render them useful. Data which have become available during the past year settle this question definitely by indicating a relatively high correlation between officers's judgments of military value and the intelligence ratings".
Army General Classification Test
- ^ abcdefghijklmnopqArmy Mental Tests. H. Holt and Company. 1920. Retrieved 2012-09-24.
- ^"History", Armed Services Vocational Aptitude Battery (ASVAB).
- ^Atwell, C. R. (1937). "Relationship of Scores and Errors on the Army Alpha Test", Journal of Applied Psychology, 21(4), 451.
- ^https://books.google.com/books?id=xhECAAAAYAAJ&printsec=frontcover&output=reader&pg=GBS.PA220, p.18
- ^https://books.google.com/books?id=xhECAAAAYAAJ&printsec=frontcover&output=reader&pg=GBS.PA220, p.19
- ^https://books.google.com/books?id=xhECAAAAYAAJ&printsec=frontcover&output=reader&pg=GBS.PA220, pp.22-23
- ^ abcdGuilford, J. (n.d). Army Alpha Examination: First Nebraska Edition.
- ^ abcSchrammel, H. E., & Brannan, C. (n.d). Army Group Examination Alpha: Schrammel-Brannan Revision.