Study Document
Automated Test Form Generation Critique of the Research Paper
Automated Test Form Generation
Critique of the Journal Article "Automated Test Form Generation" by Wim J. van der Linden and Qi Diao
Van der Linden, J.M. & Diao, Q. (2013). Integrating test-form formatting into automated test assembly. Applied Psychological Measurement, 37(5) 361 -- 374
Retrieved: http://www.utwente.nl/gw/omd/Medewerkers/artikelen/vdLinden/artikelen%20vanaf%20juni%202011/APM%202013,%20361-374.pdf
The ordering of items in test forms and the selection of test items from a database can be extremely tedious and laborious when done manually. It would be extremely advantageous to be able to do both simultaneously using automated test form generation (ATFG). But the ATFG process can be unwieldy and cumbersome given the number of variables the creator must tackle. For example, "for an item bank of size 2,000, test length of 50 items, and upper limit of the number of pages in the test form equal to 15, the number of variables in the model is equal to 1,500,000" (Van der Linden & Diao 2013: 361). According to the article, "Integrating test-form formatting into automate test assembly" from Applied Psychological Measurement, "one of the goals of this research is to find out if test-production problems of realistic sizes can be solved" and make the simultaneous generation process easier for test designers (Van der Linden & Diao 2013: 361).
The process of automated test form generation consists of two steps. The first is the selection of items from a database and ordering and formatting the item into the actual test form itself. "ATA involves the selection of the items for a test form from an item bank subject to a potentially large set of content and statistical specifications while optimizing their measurement properties" through the use of mixed integer programming (MIP)" (Van der Linden & Diao 2013: 361). While MIP is mainly used for test selection, it can also "be used to optimize the second step (automated test-form generation [ATFG]" (Van der Linden & Diao, 2013: 361). The purpose of the study is to determine how this can be accomplished in an efficient manner. The objective is to explore whether it is possible to use ATA and ATFG simultaneously for more efficient test generation and "then evaluate their performances against the case of separate models for ATA and ATFG for a series of applications with realistic item bank sizes and test lengths" (Van der Linden & Diao, 2013: 361).
Previous authors have found this quite challenging so rather than a formal literature review,…
Sample Source(s) Used
To argue that to optimize ATFG is possible, the authors simulated two different test forms: one computerized and the other paper based, according to various "content, statistical, and formatting specifications. More specifically, they used an existing set of specifications for the assembly of test forms, along with an existing item pool from a large-scale mathematics achievement testing program" (Van der Linden & Diao, 2013: 368). This was to show that ATFG was both hypothetically possible as well as applicable to the needs of an actual testing program
In their conclusion, the authors noted that it was possible to deploy a simultaneous modeling approach that allowed an integration of both selection of problems and generation of test forms, considerably streamlining existing available formats. "For future applications, the performance of the solver needs to be assessed on a case-by-case basis. The total number of decision variables is only one of the factors with an impact on the performance of MIP solvers; others are the structure and sparseness of the coefficient matrix for the constraints" (Van der Linden & Diao, 2013: 368). Thus, the simultaneous approach is recommended, based upon their calculations although this may vary according to circumstances on occasion.
While the research provided by the authors is valuable, in terms of structure, the article is somewhat problematic. First of all, the authors assume a great deal of knowledge on the part of the reader, automatically beginning with a discussion of the different forms of test-generation without setting up the reader with an extended example or grounding the problem in real-world analysis. Also, the equations used to substantiate the hypothesis are included in a separate section rather than explained. This article is thus largely unreadable by a layperson in the field and must accept the authors' conclusions on faith if they are not mathematically fluent. Even someone in the field might find it problematic that they must evaluate the author's mathematical conclusions to support or take issue with their analysis.
Related Documents
Study Document
Promising Phenomenon That Lends Itself
66).
Furthermore, social software will only increase in importance in helping organizations maintain and manage their domains of knowledge and information. When networks are enabled and flourish, their value to all users and to the organization increases as well. That increase in value is typically nonlinear, where some additions yield more than proportionate values to the organization (McCluskey and Korobow, 2009). Some of the key characteristics of social
Study Document
Management and Decision Sciences From
76). As automation increasingly assumes the more mundane and routine aspects of work of all types, Drucker was visionary in his assessment of how decisions would be made in the years to come. "In the future," said Drucker, "it was possible that all employment would be managerial in nature, and we would then have progressed from a society of labor to a society of management" (Witzel, p. 76). The
Study Document
Cross Platform Mobile and Web
82). Both desktop and Web widgets have the same basic components. Fundamentally, they use Web compatible formats, even if intended to run in a desktop environment. This means that the core of the widget is HTML and CSS code which contains the actual content of the widget, namely text, linked images/video or content pulled from a server of Web service. Alternatively, the widget content can be created using Flash, although
Study Document
Ability of an Organization to
Customer centricity then can also have a significant impact on the perspective an organization has of its market and the opportunities inherent within it and other, tangential and territory market areas as well. This aspect of blue ocean strategies being driven by customer's perspectives, preferences, unmet needs and wants further underscores its inherent value and also its usefulness from a strategy perspective. The ability to find uncontested markets, which
Study Document
How to Perform Valuations for Municipalities in South Africa
……South African Municipalities
Municipal Revenue Loss Reduction through Improved Municipal Valuation Methodologies:
Balance Sheet Enhancement of South African Municipalities to Improve Rates and Taxes Revenue Generation
Abstract
This study examines the property valuation process of Municipalities in South Africa and develops a strategy for strengthening that process in order to more efficiently value properties and ultimately to enhance municipal balance
Study Document
Application of DNA in California in Criminal Cases
DNA in Criminal Cases - Solving Cold Cases in California with Forensic Science This research will attempt to analyze and discuss the feasibility of DNA testing in solving cold cases and will study the impact that DNA fingerprinting has had on the forensic science community as a whole. DNA is generally used to solve crimes in one of two ways. In cases where a suspect is identified, a sample of that person's