Dynamic adaptive testing and peculiarities of its use with medical students

Objective: The objective of the paper is to find out the conditions for enhancing the students’ learning capacity based on the use of computerized dynamic adaptive tests in identifying the structure elements of compound objects and to give grounds for them. At present, the requirements for the results of educational institution students’ professional training have made relevant the problem of developing dynamic adaptive tests in compound objects identification that ensure conditions for self-learning. Method: The key method for the investigation of this problem is made up by the analysis of psychological and pedagogical theory and generalization of results of scientific studies by the Russian and foreign scientists in dynamic adaptive testing in education, in particular: the activity-based approach for carrying out developmental teaching under the instructional feedback; the environmental approach relying on learning with reinforcement in conditions of assessment feedback. Results: Based on the results of the theoretical research computer dynamic adaptive tests in identifying objects of anatomy have been created, the model for diagnosing the procedural characteristics of students’ learning activity has been developed. The effectiveness of dynamic adaptive testing in diagnosing the development of students’ cognitive abilities is represented and proven. Conclusion: The suggested model of dynamic adaptive tests in identifying the structure of anatomy objects allows timely diagnosing the procedural characteristics of medical students` learning activity within the educational process. It also made possible to formulate the main principles of dynamic adaptive testing based on the assessment feedback in education.


INTRODUCTION
In the present-day conditions, education is being restructured towards creating the open education.Its theoretical foundation is the pedagogy of development the origins of which are traced back to sociocultural theory of L. S. Vygotsky (1978) and Piaget's environmental approach theory (2003) (1,2).The development and self-organization of students' learning activity determine the necessity of a synergetic approach in education relying on the principles of establishing the non-linearity, instability and non-closure (3).Non-closure, or openness of the system of education implies self-agreed changes to the students' learning activity and problem environments (4).This leads to the students and problem environments mutually adapting to each other, i.e. the development of students' learning activity (adaptation) occurs accompanied by changes of the problem environment as appropriate.
The conventional, classical tests used in education are based on the closed condition that is manifested in invariableness of problem environments, including the absence of external actions on the process of testing.This conditions testing and diagnosing the results-related aspects of the students' learning activity.
As D. A. Novikov (1998) points out, in the open education systems, it is the procedural aspects of learning activity that go to the foreground.So the process of testing gains a dynamic and interactive nature and it is aimed at diagnosing the procedural characteristics of students' learning activity in conditions of teaching environments (5).
Dynamic adaptive testing implies testing has an open nature.An expert not only watches but takes an active part in the testing process that is interactive and that combines diagnosing with the process of learning.The objective of dynamic adaptive testing consists in diagnosing dynamics of the learners' development, in describing the changeability of their cognitive abilities.Consequently, unlike the conventional one, the procedure of dynamic adaptive testing is flowing and sensitive to any changes of the tested ones' learning activity.
Although dynamic adaptive testing is important in the open education, the theoretical and applied aspects of computerized dynamic adaptive tests remain underdeveloped and solving the problem of finding out the particularities of using dynamic adaptive tests in education is relevant.

LITERATURE REVIEW
The method of dynamic adaptive testing or dynamic assessment method was presented in the works of Luria (1973) (6).As defined by Haywood and Lidz, (2007), Lantolf and Poehner (2004), dynamic adaptive testing is a method of interactive diagnostics that involves planned learning of the tested ones during interaction with experts and takes into account the influence of such learning on the students' subsequent learning activity (7,8).The authors of another work describe dynamic testing as "…a procedure that takes into account the results of interference on the part of the expert" (9, p. 28).In another work the procedure of dynamic testing includes the process of learning while also bearing in mind the scope and nature of the experts' assistance (10).Dynamic assessment is interactive (11), and it diagnoses not the results but the process of learning.Dynamic adaptive testing relies on the concept of changeability of an individual (12,13).This concept indicates the importance of a human being able to change while also developing as a result of interacting with the environment (teachers, students etc. included) and adapting to the changing circumstances (14).
Thus, dynamic adaptive testing is a procedure aimed at diagnosing the process of learning, with the scope and nature of the examiner's assistance taken into account.The feature of dynamic adaptive testing consists in it making sense for the context of learning and interaction (15).As a result of interacting with students, the expert gets information about the students' having or lacking such non-intellectual factors as the ability of self-adjusting, the sense of competence, response to a challenge, criticism, a need of mastery, a need of individuality etc.The classical testing does not diagnose these parameters.
Dynamic and classical testing are distinguished from each other in their critically different purposes: not comparing students to each other but understanding, advising and designing assistance and support.Table 1 shows compared features of a traditional (or conventional) assessment procedure to the dynamic assessment procedure.
It follows from Table 1 that conventional testing is static in nature, and its expert can only observe the tested ones' activity and supervise it for the system of testing to be closed.In practice, this means that experts have to ensure the closed nature of testing conditions in order to obtain the objective data about the students' learning results.The processes of learning and conventional testing are separated in time.The objective of conventional testing consists in identifying the gaps of results of learning, and, accordingly, the procedure of testing is standardized.
The theoretic basis of dynamic adaptive testing is L. S. Vygotsky's sociocultural theory of development (1).So, the feedback has an instructional nature (15,17).In sociocultural theory, a human develops during interaction with carriers of knowledge, culture etc.
The creation of computerized dynamic adaptive tests which would play the part of an intellectual intermediary faces some difficulties that are hard to overcome (18,19,20).This is why the development of computerized dynamic adaptive tests, with instructional feedback underlying them, has remained an unresolved problem so far.Instructional feedback imposes essential limitations on the students' decision-making.At the same time, dynamic assessment of the process of identifying a compound object based on the principles of learning with external reinforcement and assessment feedback ensures conditions for free manifestation of the students' individuality in deciding what actions to opt for when looking for solutions to identification problems.In the paper suggested, the particularities of dynamic adaptive testing in conditions of assessment feedback are described using the example of medical students' learning activity in identifying structure elements of anatomy objects.

METHODOLOGICAL FRAMEWORK
In Piaget's development theory (2), self-learning or self-development of a human occurs as a result of the human's interacting with the environment.Meanwhile, learning with external reinforcement is carried out that represents one's self-learning by one's own experience of interactions with the environment.In learning with external reinforcement, feedback is assessment one in nature, i.e. each action of the tested one receives external reinforcement that can be a positive score (award) or a negative score (punishment).The objective of a learner's activity is to obtain the maximum total award for actions performed.Learning with external reinforcement implements the idea of a "hedonistic" selflearning system (21).
In this paper, it is suggested to perform dynamic assessment of medical students' process of learning in their interacting with a digital problem environment of anatomy objects by means of combining self-control of the learning activity and the external control with assessment feedback ( 22) that determines learning with external reinforcement.With regard to this, students try to receive as high award as possible while acting in a high uncertainty extent digital problem environment.If the sequence of external reinforcement or awards received after time interval  is denoted by  +1 ,  +2 ,  +3 , ..., then in the process of adaptation a learner tries to maximize the expected benefit, where benefit   is determined as a function defined over the sequence of awards.In the simplest case, profit is the total of awards: (1) Here  =  +  is the completing time interval,  is the number of the problem or episode (21).Students adapt to the digital problem environment and learn on the basis of the experience gained.The digital problem environment is a total of conditions required for exploratory efforts in the process of learning how to solve problems.The tested ones perceive the condition of the problem environment and perform actions affecting its condition.In natural intelligence systems, learning with external reinforcement is based on a learner's trying to maximize the award received while acting in a complicated problem environment with a high uncertainty level.Uncertainty of the problem environment is set by the changing conditions of solving the problems, elements of randomness introduced in parameters of the problem environment and the problems, as well as by setting a multitude of alternative options of learning actions and ways of looking for solutions to the problems.
Uncertainty causes an imbalance between the cognitive needs and cognitive capacities of students initiating exploratory efforts in them.The exploratory efforts imply the students' being able to act and look for solutions to the problems under uncertainty of the target condition of the problems.It is clear that without the learners' exploratory efforts there can be no development in the aspect of their adapting to the learning activity when mastering problem solving methods, because the changing uncertainty of the problem environment is of critical importance for the students' exploratory efforts.
In the suggested dynamic adaptive tests based on assessment feedback, students are given the opportunity to carry out their activity in solving the problems of identifying the structure elements of anatomy objects.Initially, there is the option for the learners to act in the trial-and-error method for achieving the target condition that corresponds to identification of all elements of the structure of the anatomy object.With regard to this, actions that bring one closer to the target receive a positive external reinforcement score.Actions or operations moving one away from the target are given a negative external reinforcement score.In learning with external reinforcement, two types of behavior are distinguished: the one aimed at acquisition of knowledge and the behavior based on using the knowledge available.
The digital problem environment generates problems of a certain type.Just like in the real world, each new situation (problem) is somewhat different from the previous ones.The students being trained are not informed directly as for what they have to do or what actions have to be performed.They find out what actions lead to awards based on their own experience.The students' actions are determined not only by the immediate result but also by the subsequent actions and random awards.These properties of the digital problem environment in dynamic adaptive tests -training systems (the trial-and-error method and delayed external reinforcement) are its main features.

RESULTS AND DISCUSSION
Dynamic adaptive tests are instrumental means for measuring the changes of students' learning activity as a result of looking for a problem solution.The changes are measured by obtaining data not only about the structure of the system of actions but also by identifying the total feedback coefficient is the percentage of wrong actions (23).
( 1 -the quantity of wrong actions;  0 -the total quantity of actions);   −1 -the relative frequency of external reinforcement) Index  shows the quantity of time spent as of the point of completion of the  -th task.The graph of change of the total feedback coefficient given in Figure 1 shows dynamics of the process of learning how to solve problems.Achieving the error-free activity with external reinforcement of actions does not always mean the student can find a solution to the problem independently.The error-free autonomous activity is only achieved when the student needs no external reinforcement in solving problems.
The standard interface of a dynamic adaptive test (training system) for identifying the structure elements of anatomy objects (24) (with kidney taken as an example object) is given in Figure 2. Alongside with the "distance to target" gauges and the learner's status value level, the interface facilitates the learners' actions by color-coding the correctness of matching they have completed: the numbers given correctly are highlighted in green color while the wrong ones -in red.The digital problem environment "suggests" iteratively matching the numbered elements of the structure of kidney and the names of the elements.The structure of kidney shown in Figure 2 is taken from the "Atlas of human anatomy"  by Bilich and Kryzhanovsky (2013) (25).The learners have to achieve error-free activity at rating level 10 or the status value of the identification activity corresponding to complete absence of external reinforcement of the problem environment.
When generating each new task, the kidney structure elements are numbered randomly, and the sequence of names of the kidney structure elements is formed in the random order, too.
Each action of a student conditions an attempt to identify a structural element of the anatomy object leading to one of the two events.The first event consists in the correct identification and the problem environment passes from status  to status  + 1.The second event consists in the identification being wrong and the problem environment status not changing, i.e. the transition is from  to .After that, everything is repeated until the problem environment goes to the final status  =  which is the absorbing state.For each action performed, the student gets an award the numerical value of which depends on the status of the system  and on the actions being correct or incorrect.If the action is wrong, then the award is negative and it equals  +1 = −(1 ( − )) ⁄ , with  = 0,1,2,3, . . .,  − 2  +1 = 0, at  =  − 1, as in this state, the probability of selecting a wrong action equals zero, and as a consequence, there is no award appropriate.
In case of a correct action, the award is positive and it equals  +1 = ( −  − 1) ( − 1) ⁄ , with  = 0,1,2,3, . . .,  − 2 and  +1 = 0 at  =  − 1, as in this state, the probability of the correct action equals 1.The award for a correct action in state  =  − 1 equals 0 because there is no problem of choosing the correct action.
In Figure 3, paths of the total award are given depending on the number of identification actions.It follows from the graph in Figure 3a) that initially, a student opted for the trial-and-error strategy.So the student needed 46 actions in order to complete identification of 8 structural elements.Having obtained and memorized the information, in the second episode, the student made one mistake and completed the identification of kidney structure elements in 11 actions.
The analysis of search strategies for solutions to identification problems has shown that three groups of students can be singled out.The first one, 57% of the students, initially browses the elements of the structure and tries to analyze the obtained experience of identification; they do not use the trial-and-error method as far as they can.The second group of students, 24%, is characterized by the trial-and-error method prevailing in the first fulfillment of the tasks.Having obtained and memorized the information, the students change their search strategy for solutions to identification problems abruptly.They look through the kidney structure elements attentively and find correlations in their mind.The third group -19% of the students -uses a mixed strategy, shifting from browsing and analyzing to the trial-and-error method and vice versa.
As they master the problem solving ways, the relative frequency of wrong actions decreases, which renders the actions system structure more perfect.Accordingly, the rating level of the students being trained grows (see Figure 4) and the total feedback coefficient goes down.The decrease of external reinforcement for the learners' actions is compensated by the internal information accumulated by the learners.With regard to this, the students' learning activity ceases to need external reinforcement.− self-adjusting the scope and frequency of information (passive) and active (the learner's canceling or corrective actions) actions of the digital problem environment that contribute to reducing the value of disagreement between the current and the target problem solving status (26).

CONCLUSION
The objective of external control in a dynamic adaptive test -training system -consists in encouraging such behavior of a learner that is essential for the learner's active adaptation to the changing digital problem environment of tasks.It has to be noted that a student is a system having its own analytical center and control center (27,28).Bearing in mind that a learner is a complex system, adaptation can be defined (29,30,31,32) as the process of intended change of the parameters and structure of the learner's activity (14).
Let the qualitative aspects of adaptation be discussed within the context of a learner's performing activity in the digital problem environment, beginning with the lowest, 1 st level.It should be noted that all levels are related to each other by the recurrent dependence: each change at a higher level is conditioned by completing a change cycle at lower levels and in its turn it implies correcting the activity results gained at these levels.
At the lowest information awareness change level, the learners identify the control actions accessible to them that they can perform with the problem environment and watch the responses of the problem environment.Guided by the information obtained about these responses, the learners select such tactics of interaction with the environment as to bring them to solving the problem set.As the students adapt to the parameters of the problem environment in identification problems solving, their awareness changes, they gain the skills of interacting with the problem environment and they achieve the problem solution (local objective) guided by external reinforcement.
At the next, higher, level of adaptation, the students systemize their activity and classify their actions from the standpoint of their usefulness.Information external reinforcement of activity of the students being trained in solving identification problems is of random nature.This stimulates the formation of a certain actions system structure in the students.Change of the students' actions system structure is due to inhibition of the actions that do not bring the problem solution closer.The system-forming result for which the students work when interacting with the problem environment is the structure of the learner's actions system that forms by means of decision-making (opting for actions) under complete awareness about the state of the environment.
Self-agreed change of the problem environment is associated with organization of motivation control of the learner's activity.The objective of the control center is the learners' achieving such status under which feedback equals zero, i.e. the learners' activity becoming error-free and autonomous, independent from the control center.In order to achieve autonomy, learners have to eliminate error actions and the control actions of the center from their activity.Owing to such organization of control, students being trained adapt to this problem environment in an indirect way via their own activity.
The way a learner deals with the environment, including the computer, keyboard, mouse, screen, memory etc., characterizes the learner's behavior; it is this way that enables an external observer to assess the conformity of the student's development to the control targets.

Figure 1 :
Figure 1: Change of feedback coefficient R depending on task number Z Source: authors

Figure 2 :
Figure 2: The dynamic adaptive test (training system) interface for identifying the structural elements of kidneySource:(25)

Figure 3 :
Dependence of award R on action number i. a) the first kidney structure elements identification episode; b) the second kidney structure elements identification episode Thus, dynamic adaptive testing of learning activity under assessment feedback relies on: − monitoring the learning activity, i.e. watching and logging the students' learning actions in the real-time mode; − detecting the value of disagreement between the current and the target problem solving status and correcting it via the assessment feedback mechanisms; − the system of numerical scores of learning actions, canceling or correcting the wrong actions;

Figure 4 :
Figure 4: Student's activity rating level dynamics depending on the task number Source: authors