The development by the German Transactional Analysis Association of a scientifically-based online evaluation system of transactional analysis training © 2017

The authors present the development and statistical analysis, conducted under the auspices of the German Transactional Analysis Association (DGTA), of an online evaluation system of transactional analysis training. The understanding of evaluation research is clarified, and the data-entry form and its grounding in the theory of transactional analysis are presented. Emphasis is placed on the development of the competence concept, the definition of competence categories, and the representation of the foundations of a transactionalanalytic educational theory. The scientific examination of the validity and reliability of the scales, the research process with pre-test and re-test, and the evaluation of the data in the system of online evaluation are extensively documented. In conclusion, it is claimed that this online-based DGTA evaluation is one of the few result-oriented teaching evaluation instruments in the German-speaking countries which meets scientific control criteria and is published.


Introduction
Since the establishment of the International Transactional Analysis Association (ITAA) in 1964, quality management for training in and practice of transactional analysis has been developed, differentiated, and refined worldwide.Within Europe nowadays, the European Association for Transactional Analysis (EATA) is the authoritative body, although much effort is put into ensuring that the ITAA and EATA are working to similar standards.The German Transactional Analysis Association (Deutsche Gesellschaft für Transaktionsanalyse -hereinafter referred to as the DGTA) is affiliated to EATA and is therefore the organisation within Germany that ensures that there is adherence to the worldwide standards for transactional analysis.
Since the beginning, the TA community has placed considerable emphasis on ensuring that transactional analysts are competent.For many years, the examination processes have focused on a requirement that candidates demonstrate their competence through presentation of recordings of their work with clients, accompanied by theoretical and practical discussions with a panel of internationally-accredited professional colleagues.TA students are required to engage in an ongoing process of professional supervision in order to develop the necessary range of competencies in line with the norms of the profession.Those teaching TA will typically seek 'reverse supervision' in that they expect to engage in open discussions with participants about what within the teaching is helpful or not to the students.
The project described here is one to develop a scientifically-based process of evaluation that sits alongside the ongoing interactions between teachers and students.An online system was developed so that students could provide feedback after each seminar attended, against a set of TA-specific competencies that were defined as part of the project, in such a way that students and teachers can be provided with access to their own results, and the Association can have access to summarised, anonymised results that will allow critical discourse about several aspects within quality management e.g.how varying formats of teaching are contributing to learning.
Literature Review Evaluation and Evaluation Research Hense (2006) pointed out that the practice of evaluation can be traced back to the Renaissance, with the traditions of thought underlying it going back to antiquity.It is always a matter of evaluation -even the morning glance out of the window to check the weather is an evaluation (Meyer & Höhns, 2002).Tyler (1949) was an early proponent of the evaluation of education, having conducted an eight-year study (1933)(1934)(1935)(1936)(1937)(1938)(1939)(1940)(1941) involving over 300 institutions and addressing the rigidity and narrowness of educational curricula.What became known as the 'Tyler' or 'Objective Model' incorporated questions about: defining the educational purposes/objectives; selecting useful learning experiences; organising those experiences to maximise the impact of instruction; and evaluating the effectiveness of the learning experiences.Tyler's work on evaluation, within pedagogical discussion, had led at that time to a change from input to output orientation of educational work.His work went on to form the basis of the Programme for International Student Assessment (PISA), which is a triennial international survey run by the OECD (Organisation for Economic Cooperation and Development) which aims to evaluate education systems worldwide by testing the skills and knowledge of 15-yearold students.In 2015 this involved over half a million students, representing 28 million 15-year-olds in 72 countries and economies.(OECD, 2015a(OECD, , 2015b)).
The 1960s are widely considered as the beginning of the discussion of evaluation in Europe, with Meyer & Höhns (2002) pointing out that "Compared with the USA, the professionalization of evaluation research in Europe began with a 10-year delay at the end of the sixties.The pioneers of this development included Sweden, Great Britain and the Federal Republic of Germany."(p.5).Wottawa & Theirau (1998) described evaluation as "the process of assessing the value of a product, process or a program, which does not necessarily require systematic procedures or data supported proofs for the substantiation of the judgement."(p.13) and defining evaluation research as "the explicit use of scientific research methods and techniques for the purpose of conducting an evaluation… [and which] stresses the possibility of the proof, instead of the pure claim to the value and usefulness of a certain social activity."(p.13).Reischmann (2003) defined three aspects of evaluation research as: "1.The methodical gathering and 2. the substantiated evaluation of processes and results for the 3. better understanding in shaping other practical measures within the field of education through the control of effects, management and reflection."(p.18).
Whilst instructional sessions have been evaluated within Germany since the 1960s, surveys made by different authors show that the scientific quality of evaluation procedures and of questionnaires used are rarely monitored (Döring, 2005;Schnell & Koop, 2000).Will, Winteler & Krapp (1987) echoed the problem of evaluation being used only to confirm good practice when they pointed out that, especially within the area of educational policy-making, evaluations were only to be able to better 'sell' decisions already made.Legge (1984, quoted in Hense, 2006) had referred to this as a 'crisis of use' which leads to a 'crisis of purpose'.The lack of concrete actions as a consequence of evaluation results led to the evaluation of evaluation research becoming a well-examined subsection of research and evaluation (Leviton, 2003, quoted in Hense, 2006).
Evaluation Questionnaires Donabedian (1966) pointed out that an evaluation could cover process, structure and/or result data.Process data relates to the methodology and didactics of the teaching, structural data considers the environment and/or context factors, and result data records the gain in learning.These are therefore equally dependent on the organisation of the training and on the learning capability and will-to-learn of the learner.
There were 14 scientifically-based teaching evaluation questionnaires available in German-speaking countries before this project began (Ulrich, 2013), although only two were result-oriented:
A significant consideration is the way in which, as Eisenberger and Kramer (2005) comment, criteria for evaluation are specified before the evaluation takes place, and hence are likely to have an influence on the teachings which they are evaluating.Meinefeld (2010) made similar comments, pointing out that evaluations are suited for creating their own reality when they are first introduced, so that the original concerns of the instruction are distorted.
It seemed essential, therefore, to ensure that any instrument designed for evaluating the quality of the teaching of transactional analysis must be based on those competencies that are intended to be the outputs.

Competencies
The Training and Examination Handbook (EATA, 2014) includes within it core competencies for the four different specialisation fields of transactional analysis.Individual demonstration of these is assessed during examination where candidates present live recordings of their own practice and discuss these with the panel of examiners in terms of theoretical substantiation, appropriateness to the situation, and the underlying philosophy and values of transactional analysis.This process has been happening within the TA community since the 1960s, although it is only more recently that the idea of competence in the context of training has become more generally accepted.
Even then, many non-TA accreditation processes still have the character of knowledge examinations rather than of competence evaluation.
A starting point for the use of the term competence may be seen in the work of Weber (1947) who used it within his organisational theory to mean responsibility, such that each level in the hierarchy has clearly defined responsibilities and hence its own sphere of competence.In the German language, the Latin root of the verb 'competere' means coming together, such that several factors come together so that someone has the competence for managing the situation (Vonken, 2005).The concept of competence in this sense appeared first with the American psychologist McClelland (1966) and in German-speaking countries with Chomsky (1973) writing of linguistic competence.
Since then, among the many attempts at definition, that by Weinert (2001Weinert ( , 2014) ) is often seen: "Competencies are the cognitive abilities and skills available to individuals, or which they can learn, enabling them to solve certain problems, as well as the associated motivational, volitional and social willingness and ability used to be able to use problem-solving successfully and responsibly in variable situations."(from 2014, p.27).An alternative definition provided by Erpenbeck & Rosenstiel (2007) described competencies as dispositions to selforganised behaviour; adding that "the competence concept… has a definite meaning only within the specific construction of a theory of competence" (p.20).
Within transactional analysis, candidates are expected to demonstrate competence in terms of knowledge, understanding and ability, options for action and personal motivation, intentional and effective realisation in a reflected ethical framework and an appropriate relation to context.For example, an evaluation category reads: "TA concepts are responsibly used in order to promote learning; the broad range of different learning styles and needs is considered; questions connected with the learning process are recognised and addressed in the sense of support.Feedback and evaluation are embodied in the learning process."(Criterion 6 in DGTA 2011 Training and Examinations Handbook for Transactional Analysts, p.171) To quote from Klieme &Hartig (2007), "Nuissl von Rein, Schiersmann andSiebert (2002) named 'competence development' as 'term of the year' in adult education.A current key-word search in the literature data base of the German Education Index gives 8,889 hits for competence, in the data base Psychinfo, starting from 1985, there are 27,255 hits for competence, competency and competencies -this corresponds during the entire period to three or four, in recent time even ten publications per day."(p.12).
When it comes to specifying competencies, the OECD (2005), in their Definition and Selection of Competencies website proposed (and still do): capacity to act autonomously, interacting in heterogeneous groups, and use tools interactively (e.g.language, technology).Erpenbeck & Rosenstiel (2007) worked out four competence classes: personal, activity and realisationoriented, technical-methodological, and socialcommunicative.Webler (2005) named three: social, personal and metacognitive.

Objectives
Having established that there is agreement within current evaluation research that there is no one 'correct' evaluation, and that any evaluation should be appropriate to the respective context, it was decided within the DGTA to develop its own online evaluation instrument oriented to the inherent goals of TA training, with the intention that it should provide objective and transparent data concerning the success, effectiveness and efficiency of TA training, whether of courses, modules or other formats, leading to certification or accreditation or not, and including those who do not complete training or do not take examinations.
Within this overall aim, there were four objectives: 1. that DGTA receives feedback on the TA training that it legitimises, that will contribute to quality assurance, such as through critical discussion of results at meetings of trainers; 2. that the TA teachers receive direct feedback on their own performance, can compare this to summarised results of others, and can (optionally) engage in discussions with colleagues who are also receiving feedback; 3. that participants receive feedback reports that can be used to reflect, alone or with peers, and as part of (optional) reviews with the teacher such as is customary for those in longer-term TA training; 4. to provide a vehicle for researchers to investigate specific elements associated with the impact of TA training on competencies.
In order to ensure a scientifically-based project, an agreement was made to cooperate with the Institute for practice Research and Evaluation at the Lutheran University of Applied Sciences in Nuremberg.

Ethical Considerations
It was determined that provision of input into the online evaluation, once it had been set up, would be voluntary and anonymous.Anonymity would apply to teachers and students, in that teachers would see only evaluation data relating to their own seminars, and students would see data relating only to themselves.Each participant determines their own identifying keyword, based on some rules provided so that it can be generated again if the participant forgets it, and this is used only in order to complete longitudinal tracking of a particular teacher or student.Even the generation of an identifying keyword is voluntary; participants can provide their data without this if they wish.
All data is held on a server at the Lutheran University and DGTA receives only summary data, to guarantee that there can be no conclusions drawn about individual teachers or students.DGTA has deliberately excluded itself from direct access to any raw data, so that there can be no appearance of using the results as an instrument for external monitoring.
It is intended that data might be available for further research projects.In such cases, it is possible to show which datasets originate from different courses in order to analyse differences between different course formats, but still without any identification of teachers or participants.The necessary processing and anonymisation of all data sets is under the control of the Institute for Practice Research and Evaluation at the Lutheran University of Applied Sciences in Nuremberg, and any subsequent publication of research requires the approval of the DGTA.
It is worth noting that during 2014-2015 three uses of the data were made, by students at the University of Education in Heidelberg studying for master's degrees: one examined the development of personal competence by participants-in-training and the leadership behaviour of the instructors, another examined the meaning of leadership competence within change processes and a third studied the development of TA competencies and their effects on chronic stress.

Determining the Competencies
The competencies are shown in Appendix 1: Data Entry Form.They were developed by the first author together with another Teaching & Supervising Transactional Analyst (TSTA) in the Educational field, Dr Hans Joss; Christoph Seidenfus, a TSTA in the Organisational field; and Dr Norbert Klöcker, a Certified Transactional Analyst (CTA) in the Counselling field.Each of these wrote a description and an example based on the existing EATA/ITAA competencies (EATA Training Handbook 2011).These were combined into a presentation for a teaching conference in November 2011, at which groups formed according to specialisms were asked to discuss the competence categories and their feedback was then incorporated.
As explained later in the section on Limitations, it was not possible to include someone certified in the Psychotherapy field of TA application within this project, although it is assumed that the competencies for that field will be present in the final results for transactional analysis generally.
Pre-test A pre-test was conducted during January-February 2012 when 103 participants evaluated 19 different seminars, chosen to represent as many kinds of courses as possible and with at least one event from each of the specialist groups.
In total there had been 187 participants enrolled in the events in question: average response rate was 55.1 per cent, with a range from 13-100 per cent per event.Pre-test results were not subsequently included in the statistics after the start date of 10 May 2012.
Principal component analysis (PCA) was used to examine construct validity for individual dimensions and for all items.Two components were extracted for the dimensions of personal competence and professional competence, using significance loading greater than 0.5 with less than 0.3 loading on further components (Wolff & Bacher, 2010).It was decided to remove one item from each (increasing consciousness of myself; differentiating with more confidence between harmful and supporting interventions).One component was found for each of reflection competence and relationship competence.
For PCA over all items, it became evident that the four dimensions specified in terms of content could not be proven statistically.According to the eigenvalue criterion and the Scree Plot an ideal solution consisted of five components instead of the four dimensions presented in the questionnaire.Because of this, the decision was made to re-examine the validity statistically at a later time when more datasets would be available.
Cronbach's alpha coefficient was applied for reliability, with a satisfactory value of 0.70 (Rammstedt, 2010) for all dimensions.Reliability improved slightly if an item was removed from the personal competence dimension, but as it was a slight improvement only and the item was content-relevant to DGTA, a decision was made to retain the item.

Re-test
No re-test was accomplished in the usual sense of the term -instead the pre-test process was repeated with a larger data set.By May 2014 788 participants received an invitation to replay to the Data Entry Form after it had been published in the initial Online Evaluation System Manual on 5 October 2012.39.7% of the 788 participants responded, providing 313 datasets originating from 82 seminars which had been conducted by 27 teachers.
Before the PCA, individual items were descriptively assessed and it became evident that there were a high number of missing values.A process of imputation was considered such that values would have been added (Lüdtke & Robitzsh, 2010) but was rejected because it had been possible to mark competencies as already available, or for making no evaluation if a concept did not occur within a seminar, or if the respondent was new to training and did not feel able to make an evaluation.The exclusion of all cases with missing values would have reduced the sample size to 122.Analyses of listwise and pairwise exclusions (Lüdtke & Robitzsch, 2010) were conducted; it became evident that the results were very similar so it was decided to use pairwise exclusion as this retained more cases for analysis.
As for the pre-test, PCA was applied to the individual dimensions and the four dimensions.Before the respective analyses were performed, an examination of the suitability of the correlation matrix was made, using the Kaiser-Meyer-Olkin criterion (KMO) and the Bartlett Test (Wolff & Bacher, 2010) to indicate a significant result.According to Bühner (2011), values greater than 0.8 using the KMO criterion are an indication that the data is well suited for performing a principal component analysis.Furthermore, the suitability of the individual items for the performance of a principal component analysis was examined using the MSA-coefficients.In the interpretation of these coefficients Bühner's (2011) recommendation to interpret them exactly like the KMOcoefficients was followed.There were satisfactory sample size values in all four dimensions; the correlation matrices were well suited for PCA as expressed in a good KOM-coefficient and a significant Bartlett Test.It was also evident in all four dimensions that the variables tested by the MSA-coefficients were well or very well suited for the analysis.
A comparable starting position resulted from the principal component analysis of all items.The sample size was satisfactory and the KMO-coefficient showed very good suitability of the correlation matrix for the principal component analysis.Furthermore, the Bartlett Test was significant.If one considers the individual items, then they were very well suited for the analysis (MSAcoefficient).
Since the principal component analysis is an explorative procedure, the number of components was first determined in the context of the analysis.There are various criteria for the determination of the correct number of components.However, because the determination of the number of components with only one criterion can lead to an over-or under-specification, the number of respective components was determined with the aid of eigenvalue criterion (Kaiser-Criterion), Scree Plot and Parallel Analysis.
When using the eigenvalue criterion, it is specified that all components which have an eigenvalue greater than 1 are to be extracted.With the Scree Plot the eigenvalues are represented in a diagram.In this diagram there is a kink, after which the curve becomes flatter.All components to the left of this kink are to be extracted.With the parallel analysis, eigenvalues of the empirical analysis are compared to the eigenvalues of random data.
Components are extracted when empirical eigenvalues from the analysis are greater than the eigenvalues of the random data set (Wolff & Bacher, 2010).If the different parameters for the determination of the optimal number of components came to different results, then in the context of this investigation all possible solutions were analysed, and the solution which could be interpreted best with respect to content was regarded as most suitable.
In order to assure better interpretability of the components, these are to be rotated if more than one component was found.As was already done in the pretest, an orthogonal rotation technology was used for the analyses which were performed.The Varimax rotation, which assures that the independence of the components is preserved.has the goal to produce the best simple structure possible.This means that the variables load as highly as possible on a component, and at the same time load only slightly on further extracted components (Wolff & Bacher, 2010).
As in the pre-test, in the re-test a significant loading of a variable on a component with a loading value greater than 0.5 was seen.Here one should bear in mind that the variable may load significantly on no other component at the same time; thus only values of less than 0.3 may be present here (Wolff & Bacher, 2010).
Full details of the results of principal component analysis for the individual dimensions, the exclusion of multicollinearity and difficulty artefacts, and the calculation of Cronbach's alpha for reliability can be seen in Appendix 2.

Statistical Conclusions
When one considers the analyses which were performed, the following results can be determined.
The content-related separation (operationalisation) of the items into the four dimensions (personal competence, reflection competence, professional competence and relational competence) makes sense, is valid and can be confirmed statistically.The statistical confirmation and the validity are evidenced by the univariate solution of the principal component analysis in all four dimensions.The reliability of the four dimensions is also given, since all four dimensions exhibit very good values for Cronbach's alpha.A content-related revision of the dimensions is not necessary with a view to validity and reliability, since all variables load highly on the found component and, on the other hand, Cronbach's alpha does not improve if one were to exclude individual variables of the dimensions.
In the context of the analysis it became evident at the same time that there is a "general factor TA competence", which is generated by the seminars in a homogeneous manner.This result can be justified with the univariate solution of the principal component analysis over all items.The content-related separation of the variables into the four dimensions makes sense despite the discovered general factor, since this separation is justified and assured by the contentrelated validity.
There are, therefore, very good grounds to assume the sufficient validity and reliability of the instrument.

The Competencies and TA
Personal Competence -TA related For the determination of the category of personal competence in the context of an evaluation related to transactional analysis training, we considered the need to have reference to transactional-analytic theory and conceptualisation, to correspond to the human image of transactional analysis, and to carry the developmental character which is specific to transactional analytic modelling and its working method.Item 2.1 I was able to significantly increase my ability to engage in spontaneous and situational behaviour refers to the autonomy concept of Berne, as well as to the working model (Berne, 1964;Berne, 1975).Item 2.2 I was able to significantly increase my ability for closeness in relationships ties in to the hunger for assurance, which Berne (1975) regards as a basic psychological need, and which in the work of Erskine (1998) and in relational transactional analysis is understood as the need for relationships.The attention paid to this need is regarded by transactional analysts as basic for each client and/or target-person relationship.
Item 2.3 I am significantly more able to be aware of my own feelings and deal meaningfully with them reflects the fact that feelings are a constitutive component of the system of ego states and help determine thinking, physical experiencing and the actions which follow (Berne, 2006).In Claude Steiner's (1997) concept of emotional competence the awareness of feelings takes on substantial significance in conflict resolution and stress management.
Item 2.4 I am significantly better able to distinguish between substitute feelings and feelings refers to the concept of English (1980) which makes this distinction.It takes into account the observation that many of our feelings do not stand in a cause-and-effect relationship with present experiencing, but are instead an internal psychological reaction to past experiencing and to fantasies, and are therefore often of little help for accomplishing present tasks.Item 2.5 I was significantly better able to increase my awareness of the ethical implications of my own behaviour refers to the ethical responsibility of each human being.This is a central idea in the conception of humankind held by transactional analysists, and an expression of an integrated Adult ego state.Accordingly, there are obligatory ethics guidelines for DGTA members (DGTA 2011).Item 2.6 I was able to significantly increase my ability to grasp different frames of reference and respect them uses the concept of the frame of reference from Schiff's (1975) theory.It takes into account the subjectivity of perception and thinking in the sense of constructivism, and represents a great challenge for the professional encounter.
Item 2.7 I was able to significantly increase my ability to assert myself takes up the aspect of self-assertion in various TA models.This can be seen, for example, in the concept of autonomy, in the balance of the energy distribution according to the functional model, in the construct of the integrated Adult ego, or in that of emotional competence.Self-assertion is always to be described as an 'activity,' in distinction to 'devaluation,' 'passivity' and 'passive behaviour'.
Reflection Competence -TA related Reflection competence is a metacognitive competence (Webler, 2005); a precondition for the emergence of pedagogical expertise (Neuweg, 2005); a key competence of professionalism (Combe & Kolbe, 2004); and always takes place in the relationship of practice, theory and person (Wildt, 2003).Within transactional analysis it is particularly expected within the supervision process.
Reflection breadth and depth are drawn from Leonard, Nagel, Rilm, Strittmatter-Haubold & Wengert-Richter (2010).Item 3.1 I have significantly more reflection breadth, in other words, I can regard reality from the perspective of several models refers to more than 100 models in TA theory, and the demand made of the transactional analysts to regard behavioural situations from the perspective of different models, to select an actionguiding model appropriate to the client and the situation, and to examine this choice in the process and if necessary to change it.Item 3.2 I have significantly more reflection depth, in other words, I can better create my own biographical learning history references focuses within the context of TA training primarily on the aspect of the person.It is a matter of understanding one's own behaviour as the result of earlier learning experiences and possible script decisions, and of distinguishing these from the experiencing and acting which is related to the present.In the terminology of the structural model of the personality it refers to diagnosing one's own ego states and recognising discounts.Item 3.3 I was able to significantly increase my use of feedback for my own reflection process involves special attention given to the aspect of the extent to which feedback from others can be used for reflection.That could be implicit information from the client system (e.g.resistances, justifications, tensions), or explicit feedbacks from colleagues, the supervisor or the evaluation assessment.Item 3.4 I am significantly better able to take a critical view of theories and models directs us toward the ability for critical evaluation of those models and theories which are consulted in practice in order to understand the situation, as well as for planning behaviours.Here it is a question of the competence gain to theory-practicereflection, as well as to the ability for engaging in critical discussion of theory.
Professional competence -TA related: Successful professional action requires more than extensive knowledge.Weinert (2001) proposes "general problem solving ability, critical intellectual capacity, domain-specific and comprehensive knowledge, realistic positive self-confidence and social abilities" (Weinert quoted in Henning, 2013, p.29), to which Henning (2013) adds "domain-specific strategies, routines and subroutines, personal value orientations, motivational inclinations and volitional control systems" (p.29) Item 4.1 I am significantly better able to provide appropriate diagnoses focusses on the strength of the transactional analyst to give model-driven explanations for experiences and behaviour, that are generally understandable for the addressee and also point out development steps.In this sense, it has been talked of as diagnosis and taught in the TA training.Item 4.2 I am significantly better able to create clear strategies addresses the competence to support the development steps -implicit in the diagnosis -by strategic goal-oriented action.Within the transactional analysis literature it has been referred to as process competence.
Item 4.3 I am significantly better able to recognise the possibilities and limits of contract work and to deal with them discusses the competency requirement to capture appropriately the client's ability to contract, to distinguish different types of contracts and to access the viability of contracts.
Item 4.4 I was able to significantly increase my ability to organise learning processes refers to the process competence, as well as Item 4.2, especially taking account of the aspect that self-organised processes also require contextual support, challenges, feedback and space for practicing.Item 4.5 I am significantly better able to plan goaloriented and appropriate interventions reflects goaloriented and appropriate use in practice of Berne's (1985) distinction between different types of interventions that contribute to achieving the objective of the contract in various degrees, and how this repertoire of forms of interventions has continually expanded.Item 4.6 I was able to significantly increase my ability to offer appropriate protection to the client system ties in with the three fundamental requirements for leading groups and supporting change processes: the 3 P's (permission, protection (Crossman, 1966), and potency (Steiner, 1968)).The focus at this point is on protection, which is ethically needed as well as necessary for successful practice.
Relational competence -TA related: Relational competence is seen as an essential component of professional competence (Vierzigmann, 1993;Nagel, 2001;Juul, 2005;Erpenbeck & Rosenstiel, 2007;West-Leuer, 2007;Andretta, Drexler, Pauza & Möller, 2011).It has increasingly been researched as an aspect of bonding (Gloger Tippelt, 2011) and is accorded great significance within transactional analysis.Item 5.1 I was able to significantly increase my ability to relate out of an ok attitude reflects the fact that according to Berne (1975) the attitude I'm OK -You're OK is the only constructive basis for a profitable meeting between people.Within the TA literature this attitude is generally seen as necessary while working with people in therapy, coaching, consulting or other processes of learning.Item 5.2 I have significantly more options for shaping my communication takes up on the opinion of transactional analysts that communicative competence consists of using a wide range of possibilities to support understanding and development.
Item 5.3 I am significantly better able to develop a strengthening culture of caring in social contexts responds to Steiner's (1982) proposed giving of honest and positive attention instead of destructive rules of the stroke economy.In this way the basic psychological need for attention (Berne, 1975) can be satisfied; according to the transactional point of view this is a prerequisite for being encouraged to learn and willing to change.
Item 5.4 I was able to significantly increase my ability to abstain from games and devaluations in favour of open communication and active problem solving follows the point of view of the model of time structuring that games are avoided either by objectification or intimacy (Berne, 1975).The ability to communicate openly and impartially (intimacy) enhances the chances of active problem solving.
Item 5.5 I am significantly better to remain in contact, even in relationship crises relates to Erskine's designated "contact in relationship" as a medium, "through which the process of spin-off (dissociation) can be resolved."(Erskine, 1996, p.184) and how 'staying in contact' in relationship crisis is being essentially constructive in the relationship and supports a connected feeling of self.

Additional Statements
It was recognised during the process of identification of the competencies that particular TA teachers might wish to add statements about aspects that are not represented elsewhere, such as about methods, curriculum, leader performance.The system was therefore designed to allow the inclusion of up to five optional additional statements, by teachers or by specialised groups within DGTA.The analysis of items added by teachers are seen only by that teacher and those students who have completed that version of the entry form.

Leader Performance and Seminar Organisation
The data entry form shown as Appendix 1 indicates another set of statements, related to the leadership and organisation of the courses being evaluated.These statements were not included within the statistical processes described above and will become the subject of another paper in due course.

Limitations
There is likely to have been some 'interpretation' by respondents when evaluating short rather than longerterm courses, in that an 'applies fully' claim about development of a competence over one or two days will clearly not have the same meaning as it might for a longer course.
Much has already been published, and training education provided, by the lead author.There may therefore have been an over-reliance on the input of that author, both in the way in which the competencies are understood, and within the process of the study.
Due to legal constraints within Germany, there are few TA practitioners within the Psychotherapy field of application.The authors qualifications do not, therefore, encompass this specialism.However, the competencies have clearly been linked to the therapeutic origins and later developments within the literature.

Conclusions
DGTA now has a scientific online-based evaluation system that will satisfy the requirements of the continuing education market in Germany, meet prevailing ideas about quality assurance, and provide information to the Association, the teachers and the students in ways that will allow the identification of potential improvements in how transactional analysis is being taught as well as how it is being learned.However, we must not forget that, as Dammer (2015) makes clear, empirical evaluation is potentially an instrument of domination for the control of free humans.Transactional analysts specialise in relationship and it is important that the provision of numerical values is not seen as a substitute for the discourses and open discussions that come from direct contact between teachers and students, and within professional circles.
We finish with some encouragement for others to use the system as a basis for further research projects, such as the impact of short-term versus longer-term training, whether competence increases correlate with any of the demographic data that is being collected, the impact of group sizes, whether the rate of competence development varies depending on year of study (do beginners progress faster than advanced students, or might the opposite be the case?) -and we are sure that readers will be able to think of several more exciting research opportunities.With this evaluation the DGTA would like to examine the courses with respect to their success, effectiveness and efficiency, as well as gain reference points for the effectiveness of transactional analysis as method, and asks you for your support in this.The data will be evaluated and made available to the DGTA and the director/conductor of the course.The evaluations do not permit any conclusions being drawn to your person -please also note the information regarding data security.

Norbert Nagel
The statements presented are intended to examine the extent to which your individual competence was increased by the course.You can estimate this on the basis of a six-level standard scale:

Applies fully Applies not at all
The caption on the left, e.g."applies fully", defines the meaning of the square at the far left, the caption on the right, e.g."applies not at all", specifies the meaning of the square completely on the right.The squares between them make a gradation of the evaluation.In some cases it may be that you cannot evaluate the statements, for example, if a concept did not occur at all in the course, or you stand only at the beginning of the training.Please make no cross then and leave this statement in the evaluation empty.If a competence and/or ability is already completely present for you, please mark the appropriate field beside the scale.

Information on data security
The answers contain no kind of information which can lead back to or identify you.The results of the study will be anonymously stored and evaluated.The access key to this course evaluation which you received allows no conclusions to be drawn as to your person.The directors/conductors of the courses receive none of the personal data which you enter at the end of the data entry form.The DGTA receives this data only in completely anonymised and summarized form, so that no conclusions as to your person are possible.Likewise no inferences back to its instructors are possible for the DGTA.

Information about the online data entry form
If you make an incorrect entry while filling out the data-entry form and you are not able to deactivate it, please close your browser and open the data-entry form again.When the browser is closed the entries you have made up until then will not be saved.You can have this information displayed again at any time while you are filling out the data-entry form.Please click on the button at the left and this will be displayed.
Note that the arrow keys, shift "up" (↑) and "down" (↓) shift the cross in the evaluation of a statement to the side.9. Please provide additional information about yourself.
The course-related results from this area will be evaluated anonymously and made available only to the DGTA.In order to enter the correct keyword with the next evaluation again, it should be developed as follows: In the following the results of the principal component analysis for the individual dimensions and for all items will be presented.Because only one component could be extracted in these, in conclusion the exclusion of multi-collinearity and difficulty artifacts will also be discussed.

The Personal Competence Dimension
In the dimension, 'personal competence,' all the criteria for determining the optimal number of components indicate that a component would be extracted.Also for this reason no rotation took place.The loading of the individual variables on the component can be seen in the following table : Remarks: Principal component analysis unrotated solution; KMO = 0.890; Bartlett's Test Chi ² = 693.715;p < 0.001 All variables have a high loading on the found components and this can explain 61.2% of the variance of all variables.
If one regards the commonalities, which indicate which proportion of the variance of a variable is explained by all components, then it becomes evident that these are greater than 50% with every variable.The results of the principal component analysis show that the dimension 'personal competence' has a univariate distribution.

The Reflection Competence Dimension
As already in the dimension, 'personal competence,' also in the dimension, 'reflection competence,' all criteria for determining the optimal number of components indicate that a component can be extracted.For this reason no rotation took place.The loading of the individual variables on the extracted components can be seen in the following table : 3.1.I have significantly more reflection breadth, that is, I can view reality from the perspective of several models.
.862 3.4.I am significantly better able to take a critical view of theories and models..8343.3.I was able to significantly increase my use of feedback for my own reflection process..8313.2.I have significantly more reflection depth, that is, I can create my own biographical learning history references.
.829 Eigenvalues 2.817 % of the variance of all variables 70.418 Remarks: Principal component analysis unrotated solution; KMO = 0.820; Bartlett's Test Chi ² = 414.773;p < 0.001 With the extracted components 70.4 % of the variance of all variables in this dimension can be explained.All items have high loading on the components.When one considers the variance portion of a variable, which is explained by all components (communality), it becomes evident that this is more than 60% with all variables.The results of the principal component analysis show that the dimension, 'reflection' competence" is univariate.

The Professional Competence Dimension
In the analysis of the dimension, 'professional competence,' the eigenvalue criterion, the Scree-Plot and the parallel analysis indicate that a component can be extracted.Since only one component is available, no rotation of the components was made.The loading of the individual items on the components which were found can be seen in the following All variables have a high loading on the components.These can explain 68.8% of the variance of all items of the dimension.If one regards the commonalities, which indicate which variance proportion of a variable is explained by all components, then it becomes evident that for all variables this is greater than 60%.The results of the principal component analysis show that the dimension, 'professional competence,' has a univariate distribution.
The Relational Competence Dimension Also for the last of the dimensions of competence increase to be investigated, 'relational competence', the criteria for an optimal number of components indicated that a component was to be extracted.Therefore, again no rotation was made.In the following It became evident that all variables had a high loading on the found component, and that these can explain 74.5% of the variance of all variables.If one regards the commonalities, it is evident that for all variables the variance proportion which can explain all components is greater than 60%.The results of the principal component analysis show that the dimension 'personal competence' has a univariate distribution.

All Items
The analysis of all items with a principal component analysis was intended to test whether or not the four dimensions in which the items were separated in terms of content could also be represented statistically.The number of components to be extracted was determined in turn with the eigenvalue criterion, the Scree Plot and the parallel analysis.In the diagram below the course of the eigenvalue for the empirical data (line with x), and for the random data of the parallel analysis (dotted line) can be seen.
Here it becomes evident that, according to the eigenvalue criterion, two components are to be extracted, since these have an eigenvalue greater than one.However, according to the Scree Plot and the parallel analysis, only one component is to be extracted.
Because the drop in eigenvalue between the first component (eigenvalue = 12.535) and the second component (eigenvalue = 1.099) is extremely high, this also speaks for the extraction of only one component.This is so especially because the second component barely attains an eigenvalue greater than one.Since, also according to the scree plot and the parallel analysis, only one component is to be extracted, the principal component analysis was computed with only one component.It became evident that all variables have a high loading on the found component and these can explain 57.0 % of the variance of all variables.When considering the commonalities it was noted that the variables 2.4, 2.7 and 3.2 have a value less than 0.5.For all other variables a variance proportion of more than 50 % can be explained by all components.Even if the variance explanation for variables 2.4, 2.7 and 3.2 by the component is not optimal, the extraction of a component presents the best solution of the principal component analysis.
In this respect it could be stated that the construct 'measurement of competence increase' is also univariate and there is a general factor, 'TA-competence'.However, also this found general factor does not speak against the contentrelated separation of the items which was specified by the operationalization, because the separation is assured by content-related validity.
In order to ensure the solution with one component further, the principal component analysis of all items was also more exactly analysed with two components and the Varimax rotation.As already described, the eigenvalue criterion suggested a solution with two components.However, it became evident here that this solution cannot be interpreted meaningfully with respect to content.Only three variables of the first component and two variables of the second component could be clearly assigned.All other variables load on both components.
Furthermore, in order to attain additional assurance with the two-component analysis the rotation method was changed.For this the oblique rotation method, Promax, was selected.The special characteristic of oblique rotation techniques is the fact that the components may correlate with one another (Wolff & Bacher, 2010).With the Varimax rotation used before, which is an orthogonal rotation technology, the components may not correlate with one another.It became evident that with the performance of the principal component analysis using the Promax rotation and the extraction of two components, the variables can be assigned more definitely to one of the components.(With the Promax-Rotation a Kappa of 4 is used.This Kappa determines how strongly the components may correlate with each other.Values between 2 and 4 are recommended (Wolff & Bacher, 2010).)With this solution it became evident in part that a personal component and a professional component can be extracted.However, the correlation between the two components was 0.767, which corresponds to a very high connection.However, with an oblique rotation technology the correlation between the components should not be too high, since otherwise the components can only be differentiated with difficulty (Wolff & Bacher, 2010).For this reason the solution using a Promax rotation was rejected.

Exclusion of Multicollinearity
Since all items exhibit a high loading on the first component, it was examined whether a multi-collinearity of the variables can be excluded.Multicollinearity represents a, "reciprocal dependence of variables in the context of multivariate procedures," (Bortz& Schuster, 2010, p. 583), which can falsify the result of the principal component analysis.Since the correlation of the variables among themselves is a basic condition for the performance of a principal component analysis, attention should be given that the correlations are not too high.If very high correlations are present, this is an indication of multicollinearity (Schendera, 2010).The correlations among the variables were tested for the principal component analyses which were performed.These lay between greater than 0.3 and less than 0.8.Since very high coefficients of correlation were excluded thereby, it can also be assumed that no multicollinearity is present with the computations performed.

Exclusion of Difficulty Artifacts
In the context of the principal component analysis of all items it was also investigated whether difficulty artefacts are present, which are problematic for the analysis.Difficulty artefacts are present if the correlations are systematically distorted by strongly varying item difficulties (Wolff & Bacher, 2010).Since the basis of the principal component analysis is the correlations, falsified results can occur.In order to be able to exclude difficulty artefacts, the procedure according to Bacher, Pöge & Wenzig (2010) was selected.Here the unrotated component matrix is analysed more exactly.If a distortion caused by difficulty artifacts is present, the variables load on the first component positively and the loadings of the second component correlate with the degree of difficulty of the items (Bacher et al., 2010).
To conduct this analysis, the degree of difficulty for the examined variables of the four dimensions had to be computed.
Here the formula according to Dahl (1971) was used, by which the sum of the points reached by a variable is divided by the maximum point sum reachable by the variables.Attention was given that the lowest category ("not at all" on the available scale) was coded with zero (Bortz & Döring, 2006).This degree of difficulty obtained was then correlated with the loadings of the variables on the second unrotated component.For this the principal component analysis with a 2component solution was used.Here it becomes evident that the correlation between the degree of difficulty and the loading of the variables on the second unrotated component (r = 0.30) is weakly pronounced.
To this extent it can be stated that although all variables load positively on the first component, nonetheless, because of the small correlation between the second unrotated component and the difficulty degree, a distortion of the results by difficulty artefacts can be ruled out.

Calculation of Cronbach's alpha as Index for Reliability
Now that the results of the principal component analysis for the statistical verification of the construct validity have been presented in more detail, in the following the results of the reliability analysis will be elaborated.A satisfactory reliability is present with values for Cronbach's alpha greater than 0.70, and values greater than 0.80 are considered good reliability values.(Rammstedt, 2010).
In the following tables the reliability values for the individual dimensions are presented.In all dimensions the values for Cronbach's alpha were greater than 0.80 and so the reliability can be considered to be good.Furthermore, it can be gathered from the tables how the Cronbach's alpha changes when a specific item is eliminated.This value is presented in the sixth column of the table.If this value of Cronbach's alpha is greater than the value for the complete effective dimension, this is a sign that this item should be removed from the questionnaire in order to achieve better reliability.However, in all dimensions it became evident that reliability, measured with Cronbach's alpha, is not improved when an individual item is eliminated.
Teaching & Supervising Transactional Analyst (Educational) is lecturer in education sciences at the University of Education, Heidelberg and directs the Institut für Integrative Pädagogik und Erwachsenenbildung in Neckargemünd by Heidelberg.He can be contacted at TASeminare@IPE-Nagel.de Joachim König, DPhil is Vice-President of the Lutheran University of Applied Sciences, Nuremberg and the Director of the Institute for Practice Research and Evaluation there.Sebastian Ottmann MA is Research Assistant at the Institute for Practice Research and Evaluation of the Lutheran University of Applied Sciences, Nuremberg.Annika Hahnle MA is in advanced TA training and is trainer at ubf management consultancy, Tübingen Appendix 1: Data Entry Form Editor's Note: formatting changed to save space and for IJTARP pagination Thank you for your participation in the training evaluation of the DGTA.The following training session you attended will be evaluated with this data entry form: Title of the course _______ (entered automatically by the system) Form of the course _______ (entered automatically by the system) Leader of the course _______ (entered automatically by the system)Period under consideration from ____ to ____ (entered automatically by the system) indicate the kind of participation in TA training: which training year are you at the moment?______________________9.3.2Approximately how many courses have you completed?___________________The DGTA is also interested in recording the long-term development of individual participants.For this reason you can in the following indicate a keyword, which you again indicate for following course evaluations of the DGTA.It is not possible to make inferences as to your person and the change is evaluated only in anonymised form.
(If you are not able to evaluate one of the following statements, please make no entry.) 7. In conclusion, please evaluate the statements related to the leadership and organization of the course.The course-related results from this area will be evaluated anonymously and made available only to the director/conductor of the course.
Number of your brothers and sisters, e.g.: 1  The first two letters of your place of birth, e.g.: RH Note: If you are no longer certain, which keyword you have used for an earlier questioning you can enter here your keyword and test it.(Opens a new popup window.Please deactivate your popup blocker for this page if necessary.) your date of birth, e.g.: 06The last two letters of the first name of your mother, e.g.: UN  table the loading of the individual items on the component may be seen: Since only one component was given, there was no rotation.The loading of the individual items on this component can be seen in the following table.