knowledge_assessment:q-matrix
Differences
This shows you the differences between two versions of the page.
| Both sides previous revision Previous revision Next revision | Previous revision | ||
|
knowledge_assessment:q-matrix [2012/07/11 11:02] jpetrovic [How do I create a q-matrx?] |
knowledge_assessment:q-matrix [2023/06/19 18:03] (current) |
||
|---|---|---|---|
| Line 27: | Line 27: | ||
| The goal of q-matrix construction is to extract underlying, or latent, variables, which account for students' differential performance on questions. A q-matrix can be created in two ways: | The goal of q-matrix construction is to extract underlying, or latent, variables, which account for students' differential performance on questions. A q-matrix can be created in two ways: | ||
| - | * by having the domain experts analyze assessment items (questions) and define the concept (often referred to as attribute) corresponding to each item and thereby construct the q-matrix. In this case all the concepts in the q-matrix are exactly defined and labeled as elements of the domain. Q_matrix and student's knowledge is therefore easily interpretable. Problems: "//a q-matrix is a much more abstract measure of the relationships of questions to concepts. We might assume that the questions designed to test students are a more accurate reflection of the teaching objectives than an abstract construct which relates questions to underlying concepts.//" | + | * by having the domain experts analyze assessment items (questions) and define the concept (often referred to as attribute) corresponding to each item and thereby construct the q-matrix. In this case all the concepts in the q-matrix are exactly defined and labeled as elements of the domain. Q_matrix and student's knowledge is therefore easily interpretable. This method relies on a simple hill-climbing algorithm that creates a matrix representing relationships between concepts and questions directly. The algorithm varies c, the number of concepts, and the values in the q-matrix, minimizing the total error for all students for a given set of n questions. To avoid of local minima, each hill-climbing search is seeded with different random Q-matrices and the best of these is kept. Problem: "//a q-matrix is a much more abstract measure of the relationships of questions to concepts. We might assume that the questions designed to test students are a more accurate reflection of the teaching objectives than an abstract construct which relates questions to underlying concepts.//" |
| * by automatically extracting the matrix directly from obtaines students' performance on the test items. in this case, the number of concepts is determined by the algorithm (and is usually much smaller than the number of assessment items) and their labels are automatically assigned (and do not reffer to exact domain concepts). Experts can examine the resulting q-matrix to ensure that the extracted relationships seem to be valid, and then use that q-matrix to guide the generation of new problems. | * by automatically extracting the matrix directly from obtaines students' performance on the test items. in this case, the number of concepts is determined by the algorithm (and is usually much smaller than the number of assessment items) and their labels are automatically assigned (and do not reffer to exact domain concepts). Experts can examine the resulting q-matrix to ensure that the extracted relationships seem to be valid, and then use that q-matrix to guide the generation of new problems. | ||
| + | ==== Factor analysis ==== | ||
| - | ((see Ham85 for a discussion of item-response theory)) | + | Factor analysis can be considered as an alternative to q-matrices. Concepts are in that case automatically determined using covariance matrix. Number of concepts should be smaller than number of questions. Still, this methos has proven to be less fault tollerant. Still, when forming a correlation matrix, we lose individual student data in favor of calculating average relationships between questions. The q-matrix method is optimized to assign each student the most appropriate knowledge state, using all available response data for each student. |
| - | Factor analysis: | + | ==== Discussion ==== |
| - | How to automatically determine concepts? Using covariance matrix. Number of concepts should be smaller than number of questions. Still, this methos has proven to be less fault tollerant. | + | |
| - | ==== Q-matrix method ==== | + | As Sellers found in her research, the results obtained through q-matrix analysis seem to describe relationships among variables in interpretable ways. Factor analysis and principal components analysis, on the other hand, do not readily offer interpretable results. |
| - | The q-matrix method is a simple hill-climbing algorithm that creates a matrix | + | Later researchers found that, although the q-matrix model was a good way to compare student data to a concept model, expert-constructed q-matrices did not correspond to student data any better than random q-matrices did. |
| - | representing relationships between concepts and questions directly. The algorithm varies | + | |
| - | c, the number of concepts, and the values in the q-matrix, minimizing the total error for | + | |
| - | all students for a given set of n questions. To avoid of local minima, each hill-climbing | + | |
| - | search is seeded with different random Q-matrices and the best of these is kept. | + | |
| - | When forming a correlation matrix, we lose individual student data in favor of calculating average relationships between questions. The q-matrix method is optimized to assign each student the most appropriate knowledge state, using all available response data for each student. | + | The findings in Brewer's previous research, which found that the factor analysis method performed poorly in comparison with the q-matrix method when fewer observations were available. |
| - | As Sellers found in her research, the results obtained through | + | ==== Literature ==== |
| - | q-matrix analysis seem to describe relationships among variables in interpretable ways. | + | |
| - | Factor analysis and principal components analysis, on the other hand, do not readily offer | + | |
| - | interpretable results. | + | |
| - | Later researchers found that, although the q-matrix model was a good way to compare student data to a concept model, expert-constructed q-matrices did not correspond to student data any better than random q-matrices did. | + | [[http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=4958804&tag=1|Shuqun, Yang, Cai Shenzheng, Yao Zhiqiang, and Ding Shuliang. “The Concept Lattices of Q-Matrices.” In First International Workshop on Education Technology and Computer Science, 2009. ETCS ’09, 1:413 –417, 2009. doi:10.1109/ETCS.2009.101.]] |
| + | |||
| + | [[http://coitweb.uncc.edu/~tbarnes2/itsfall04/papers/acm.pdf|Barnes, Bitzer. Fault Tolerant Teaching and Automated Knowledge Assessment.]] | ||
| + | |||
| + | [[http://repository.lib.ncsu.edu/ir/bitstream/1840.16/4612/1/etd.pdf|Barnes, Tiffany Michelle. “The Q-Matrix Method of Fault-Tolerant Teaching in Knowledge Assessment and Data Mining.” North Carolina State University, 2003.]] | ||
| + | |||
| + | [[http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.112.9876&rep=rep1&type=pdf|Barnes, M. T. The Q-Matrix Method: Mining Student Response Data for Knowledge.]] | ||
| - | the findings in Brewer's previous research, which found that the factor analysis method performed poorly in comparison with the q-matrix | ||
| - | method when fewer observations were available. | ||
knowledge_assessment/q-matrix.1341997371.txt.gz · Last modified: 2023/06/19 17:49 (external edit)
Except where otherwise noted, content on this wiki is licensed under the following license: CC Attribution-Share Alike 4.0 International
