TEACHING EXCELLENCE DATABASE

While legal teaching

Methodology

I – Sources of inspiration and the contemporary deficits of the traditional evaluation method Legal education in Hungary is particularly theory-based: according to the national requirements on higher education training and output this ratio may be as high as 70-80% (see Annex 4, section V.2.5.2 to 18/2016 EMMI decree). The traditional method of evaluation consequently assesses the theoretical knowledge of the students. Evaluation is based on a number of pre-defined topics or questions, which the students may learn in advance and at the exam they are expected to give the correct answer to a random set of those questions. There are a number of concerns with regards to the focus points of this traditional methodology: first, the room for student creativity is limited; second, in case of systemic application it easily leads students to the false assumption that there is always one single correct answer to legal questions, third, with the advancement of information technology this method seems to demand a superfluous recital of the class material; and fourth mere recital of pre-assigned class material becomes prone to unfair use of new technologies. Literature reflects there is an ongoing discussion on this and other similar problems in the English speaking legal education (see e.g. Carel Stoker, Rethinking the Law School, CUP, 2015; Paul Maharg, Transforming Legal Education, Routledge, 2016) but this discussion seems less present in Central-Eastern European realities. The staff involved in legal teaching activity had mostly received legal education only: lawyers are teaching future lawyers. Realising this inherent feature of legal education and curios whether this feature was internationally dominant or not, our department organised an international workshop in 2016 with the aim of initiating a dialogue with foreign partners on identifying common problems and exchanging good practices. Teachers from all carreer phases coming from inter alia from Austria, Slovenia, Poland, Norway, France defined two particular challenges first, the difficulty of teaching essential legal skills, predominantly the skill of deconstructing normative and descriptive texts (basically legal norms and facts of a case) into simple premises ready for logical reconstruction. It remained undisputed that due to the social role of law and lawyers, this competence building cannot be achieved in legal education by means of sacrificing the need for a relatively large, abstract knowledge of the law. A second recurrent issue in reports countries referred to the challenge of motivating the students. As head of our Law Schoolʼs Excellence Program I have been organising individual traineeships to a selected few students of the university, whereas the students spent about six month as a trainee in a judicial, governmental or other institution, under the supervision of a mentor, who has been a senior employee of that particular body. Feedback from these mentors and students often pointed to the inherent difficulties of getting across the practical application of the purely academic knowledge. I have been preparing students to international moot court competitions. In these gamified environments there judges who are usually outstanding legal experts representing multiple legal cultures and consequently they form their opinions based upon a wide variety of legal education methods. Adapting their comments and feedbacks helped me to clarify certain methodical errors in our preparations, and looking for ways to correct them. II – Goals for the new assessment mechanism Based upon the above inspirations I defined the following goals for the system of evaluation applied for law students at their mandatory public international law course. The evaluation needed to find a balance between the necessary academic knowledge while supporting essential skill development. The exam needs to encourage a learning process that is aimed at mitigating the overwhelmingly theoretical nature of the legal education. The exam should mimic lifelike conditions. First, as most future clients are not lawyers themselves, it is essential to face facts presented by lay persons. An important criterium of the evaluation is therefore the correct legal classification of the events presented. The exam must be open-book to provide access to the students to every relevant source they would be able to use as a lawyer meeting a client. Critical analytical skill are therefore also part of the evaluation. The new assessment required prior adjustment of the training program itself. The exam method was offered as an alternative to the traditional system initially in the Fall semester of 2019 and with the lockdown in 2020 it became the norm. The pandemic defined new goals. The evaluation needed to be made socially distant, which was relatively easy to do with the incorporation of IT technologies. The introduction of IT in a mandatory course in legal education directly led to the issue of equal opportunities, so we needed to make the exam accessible-for-all, regardless of age, disabilities, and personal conditions, with particular reference to the part-time students, who are usually studying along their normal day jobs. III – Implementation and review The exam is composed of two parts. First, there is anáá entry test that is conducted electronically and gives instant feedback to the students on their success. This test measures academic knowledge, with a goal of defining a minimum threshold. After prior tests with volunteering students, we decided to open a twenty minutes period to solve twenty questions. The threshold is 55%, those who pass it (70-90% of students daily) can turn to the second phase of the exam. The second part is an open-book case assessment. Two cases are presented to the students and they may freely choose which one they intend to solve. The case facts are from 200-400 word long extracts from news agency reports, or recent newspaper articles describing an event relevant to the class material. The evaluation of the case assessments are conducted by two teachers to mitigate the subjective aspect. In case their grade shows an unusual difference (2 grades out of 5) a third teacher also takes a vote. Otherwise the students final grade is the average of the grades received from the two evaluators. Failure means either an unsuccessful entry test or the unilateral decision of at least two teachers of the department. IV – Persons involved and role allocation Many partners of the department were asked to provide ideas and tips for the case assessment scenarios. Several colleagues contributed anonymously from the Ministry of Foreign Affairs and from the Constitutional Court. Senior teachers of the Department gather and finalise the case assessments, also from Hungarian and international news agency reports. The head of the department provide substantive guidance in a written memo to the actual evaluators in order to achieve consequent grading. Every person involved in the teaching work is eligible to become evaluators – otherwise the number of test would be overwhelming. This means Dr. Tamas Adany – associate professor, head of Department Dr. Reka Varga – associate professor, deputy head of Department Dr. Istvan Sabjanics – senior lecturer Dr. Lilla Munkacsi, Dr. Nora Ronyai – visiting lecturers Dr. Zeno Suller, Dr. Gabor Galambosi – PhD students.

Tools, equipment, technology used

The assessment mechanism was designed bearing in mind the specificities of the applicant's law school. The overall number of students enrolled is around 500 annually, 300 as full time students and around 200 registered as part time students. This later group is diverse in terms of age, and their significant majority leads a professional career in another day job. Inclusivity therefore demanded to refrain from too complex or specialised technical equipment. Legal education traditionally does not use too many of such solutions – therefore these are alien from students. A notable exception is the special legal databases, and improving knowledge in their use turned out to be a particular benefit of this new method. For the first part of the exam Neptun Unipoll is used. Unipoll offers a possibility to build up a pool of questions (currently around 150-200 per semester) and it generates an individual test for each student registered to the current exam. As the correct answers are previously marked in the system, evaluation is automatic and instantaneous. For the case assessment itself basic software, a browser, a text editor and an internet access is satisfactory. Students are also advised to activate their university mailing accounts as it gives them access to a number of paywall protected databases. The case of the day is published at Departmentʼs website as a main rule. Two alternatives have been regularly used: first there is MS Teams, which is the dedicated DL software of the Law School. Shared Dropbox folders often had to be the fallback plan. Students are required to send their assessment attached to an e-mail sent to a dedicated address of the department, and to use a specific file naming structure. The e-mail attachments are batch converted to Adobe Pdf format using a Linux bash script and Libre Office. Discussion of the grades is subject to a separate meeting at Teams, and students concerns are also addressed by e-mails.

Outcomes and outputs, main results

To date more than 1000 exams have been passed under this new mechanism. Statistical data suggest that exam results have improved compared to the previous method. In the 2019/20 academic year the first semester exams followed the previous system with an oral exam for all students. The new mechanism was applied to the same student groups in the second semester. More full and part time students attempted and passed the exam (success rate raised 81% to 90%, and 67% to 81% respectively) but in turn averages have decreased somewhat. After the novelty of the system have subsided, students feedback do not show significant difference from any other previous semester. The new method offers better selection criteria for students nominated to various aspects of the excellence program. There was a rise also in the number of students opting for our department to conduct their mandatory research project for interim papers.

Lessons learnt

This method feels more work-intensive than the previous assessment method, but not in terms of working hours. Both for the drafting and grading phases more creativity and attention is required, this system cannot be operated reflexively. Also it is important to avoid subjectivity in grading as much as possible. In order to achieve that every group of exams is graded independently by two evaluators. If their proposed grades have a difference of 2 or more, a third evaluator takes a vote. It is important that an entire group of exams is graded by the same pair of evaluators since that is the only way to notice plagiarism. While guidance is offered to the evaluators it is impossible to expect one single correct solution, there are always unforeseen, nonetheless correct answers. The new method in one year resulted in more active student participation in class work. Optimization of the number of students taking the exam at one time needs further attempts. The more exam dates are offered, the less exams need to be corrected per occasion. However it requires more cases to be drafted in advance. Grading more than a 100 exams files within a week for an entire month is demanding. It is also necessary that the exams are overseen by a senior fellow of the department, although full automatization is theoretically possible. Technical and other difficulties however demand immediate reaction for about 1% of the students.

Adaptability and sustainability of the best practice (for other institutions)

The method is directly applicable to almost every legal education programs and subjects except for entirely philosophical or historical courses. International law is highly specialised therefore the knowledge to correctly solve these cases is not readily or easily available in daily praxis. With less specialised courses it may be necessary to pay attention to identifying and overseeing student work. For international law programs it may be relevant that this evaluation method is completely location independent: therefore it can help facilitating cooperation among partners from all types of higher education institutions and cover a broad geographic scope across Europe.

Promotion of best practice

The method has not been promoted yet.

Scope and impact

• Course/department level
• Faculty level
• National level
• EU/EHEA/International level

The method can be relevant at the national level, because this evaluation method helps students to be able to use correct legal terminology; improves their writing skills; demands clear and logical argumentation; enables students to formulate legally relevant and justified opinions; enhances students skills to select and use relevant information; and improves student skills in using specialized legal databases (HUDOC, UN Treaty Collection, UNMICT Case Law, etc). At a European level such a competence oriented evaluation in legal education poses numerous benefits in terms of service to the society. The method highlights the importance of critical thinking and creativity and this way enables future lawyers to build sustainable careers and become active citizens. This mechanism supports law professors in implementing competence-based evaluation without giving up on other, knowledge-based requirements.

6.1 Digitalization

• Outstanding, innovative, excellent practices of online / blended / hybrid learning
• Digital skills development and assessment both general and profession-related, embedded in course design, in teaching and assessment
• Digital scientific sources used in T&L

Reasoning: This project intends to import distance learning in legal education through the example of international law.

---

6.2 Internationalization

• Other:

  The course content is essentially international.

Reasoning: The evaluation encourages law students not only to consider but to correctly find and use relevant legal sources outside their own domestic legal framework. International law is often perceived as something exotic and distant system, while the method subject to this project helps to bring it closer to ordinary lawyers of the future.

---

6.3 Inclusion and diversity, universal design

• Inclusive course design, universally designed divers activities to meet special students' needs
• Alternative, flexible assessment methods for students with special needs

Reasoning: Students were encouraged to seek out teachers if they need technical assistance or if they require support in obtaining the necessary technical apparatus (laptops, etc.) Students with disabilities can pass the exam in a way befitting their individual conditions. People with motoric challenges in their hand or sight impaired may present their arguments orally after the given amount of time. People with dyslexia can have a prolonged preparation time

---

6.4 Sustainability

• Sustainability goals are addressed in the course(s)
• Teaching material contains profession related sustainability aspects

Reasoning: The UN SDG as such is part of the course subject to the evaluation method. Goals 4, 5, 10, 16 and 17 are addressed in details as well as migration and environmental protection.

3.3 Public contact datas

Name	Email address	Website
Tamas Adany	adany.tamas@jak.ppke.hu	http://jak.ppke.hu/nemzetkozi-kozjogi-tanszek