ResearchPDF Available

Judging the Judges: Measuring Performance of District Court Judges in Slovakia

Authors:

Abstract and Figures

Slovak judiciary has been in recent years considered corrupt, untrustworthy and in general unsatisfactory both in perception and objective measures. In contrast, since 2011 it functions in an environment of unprecedented transparency. This analysis aims to utilize available descriptive data about performance of individual judges to evaluate and compare judges and identify some of the reasons for inadequate performance of the system. Performance is measured on two dimensions: quality and efficiency with eight indicators. Subsequent cumulative link mixed model analysis provides answers to four questions asked in the research: a) do judges matter for the performance of judiciary, and what other factors contribute?; b) are there courts doing significantly better or worse than others with regard to quality and efficiency?; c) can we identify individual effect of judges and separate good ones from bad ones?; d) finally, what other factors can explain the variance in performance? In general, the analysis attempts to contribute to better understanding of Slovak judiciary, however the overall meaning of the proposed measurement needs to be established in the future via interviews or different types of data collection, such as surveys or qualitative analysis of judicial decisions.
Content may be subject to copyright.
1
Judging the Judges:
Measuring Performance of District Court Judges in Slovakia
Samuel Spáč, Department of Political Science, Faculty of Arts, Comenius University in Bratislava
samuel.spac@gmail.com
Erik Láštic, Department of Political Science, Faculty of Arts, Comenius University in Bratislava
lastic@fphil.uniba.sk
Abstract:
Slovak judiciary has been in recent years considered corrupt, untrustworthy and in general
unsatisfactory both in perception and objective measures. In contrast, since 2011 it functions in an
environment of unprecedented transparency. This analysis aims to utilize available descriptive
data about performance of individual judges to evaluate and compare judges and identify some
of the reasons for inadequate performance of the system. Performance is measured on two
dimensions: quality and efficiency with eight indicators. Subsequent cumulative link mixed model
analysis provides answers to four questions asked in the research: a) do judges matter for the
performance of judiciary, and what other factors contribute?; b) are there courts doing
significantly better or worse than others with regard to quality and efficiency?; c) can we identify
individual effect of judges and separate good ones from bad ones?; d) finally, what other factors
can explain the variance in performance? In general, the analysis attempts to contribute to better
understanding of Slovak judiciary, however the overall meaning of the proposed measurement
needs to be established in the future via interviews or different types of data collection, such as
surveys or qualitative analysis of judicial decisions.
Key words: judicial systems, accountability, performance evaluation, transparency
Introduction
In recent years Slovak judiciary has been considered as untrustworthy (TASR 2012), ineffective
(Schwab and Sala-i-Martin 2013), corrupted (Transparency International 2013), or in "disarray
and turmoil" (Bojarski and Stemker Köster 2012). To some extent this has been caused by
various highly salient issues, such as numerous judicial decisions regarding defamation of the
former President of the Supreme Court and the Judicial Council Štefan Harabin, disciplinary
2
procedures against judges who stood up against ongoing practices within the judiciary, or
"discrimination lawsuits" of considerable amount of judges against the state. However, dire
perception is not the only concern that indicates problems in the functioning of Slovak judicial
system. According to CEPEJ report (2014) on the efficiency of European judiciaries, the Slovak
judicial system seems to be among five worst with regard to most important indicators of
efficiency - clearance rate, and the disposition time of cases.
In 2011 the unsatisfactory situation led to a series of reforms considerably increasing the level
of transparency in the judiciary initiated by Minister Lucia Žitňanská. Currently, basically all
judicial decisions should be published; public can find information about selection procedures,
including CVs, motivation letters and information about family connections in the judicial system
for all candidates; judges' annual asset declarations; as well as annual descriptive information
about performance of individual judges. Regarding massive transparency changes across the
globe Krastev (2013) raises two major critiques: first of all, inundating public with
unprecedented volume of information does not contribute to the idea of informed citizens;
secondly, high levels of transparency leads to cherry-picking of cases by media and politicians,
and actually adds to raising levels of distrust towards democratic institutions. In this paper we
aim to challenge these ideas and argue that increased transparency can actually help to the
improved targeting of problems that can be consequently addressed with much more precision,
but also to increase personal accountability of judges and contribute to merit-based decisions
with regard to judicial careers, hence in the long run contribute to the recovery of lost trust.
At the heart of this research lie individual judges which is not common to study of courts and
judiciaries. The main reason for that is probably the lack of data. In addition, poor performance
of judicial systems tend to be generally explained by insufficient resources, both material and
personal, complicated legal orders, high number of incoming cases, while the role of judges
remains downplayed. Descriptive data about performance of Slovak judges can be found on the
website of the Ministry of Justice in annual statistical reports, however for this analysis we will
use data that were provided by the Ministry in a single dataset. For the purpose of analysis we
will focus on judges in ordinary judiciary only, and specifically on district court judges, hence
exclude judges from regional courts, Specialized criminal court, and Supreme court. we plan to
utilize the data about judges' performance in a two-step process. First, we will introduce a
measurement of judges' performance on two dimensions: quality and efficiency; and
subsequently we will use the data for statistical analysis in an attempt to provide answers to two
main research questions: a) do individual judges matter for the performance of judiciary?; b)
and, are there other factors that can explain the variance of the measured performance, such as
judge's agenda, gender, or family connections in the judiciary? In addition, if the results will
3
show that courts and judges, possibly a region, contribute to the observed variance, can we
identify individual effects they have?
1.Evaluating judges: the starting point
Performance evaluation of judges is not a completely new idea in the Slovak context. The
legislative amendment from 2011 added to the Act on Judges No. 385/2011 that evaluation shall
take place at least every five years and a judge should be evaluated with respect to opinions of
appellate senates about her judicial decisions and abilities, with respect to the number of
unresolved cases, delays in proceedings and their causes, and even with respect to survey about
her work evaluating fluency and "dignity" of her judicial proceedings. After the process judge
can be evaluated as either "excellent", "good", or "unsatisfactory". The last can be a ground for
the initiation of disciplinary procedures, which, do not seem to be functioning effectively in
Slovakia.
The method for evaluation of judges proposed in this paper goes beyond evaluation procedure
as is entrenched in the law, and also beyond comparable measures available in the literature.
Moreover, literature discussing judicial evaluations does not seem to be numerous. In the US,
judges in many states undergo some evaluation process that should play a role in elections, open
or retention, that is necessary in order for them to remain in the office. Reactions of judges, as
well as their assessment of existing measures in the literature vary. On one hand, judges
appeared to understand the public's need to be informed and consider it
1
a legitimate tool
(Sterling, Stott and Weller 1981), on the other, they were generally opposed to outside criticism
(Flanders 1978). Majority of performance evaluations done and discussed in the literature was
based on surveys among legal professionals and general public that holds stake in evaluation of
any particular judge. Nevertheless, there has been attempts to evaluate judges using descriptive
statistics about their performance, such as used citations outside their circuit (Choi and Gulati
2004) and the success rate at the appellate level (Cross and Lindquist 2009).
There are currently approximately 1.400 judges working on 64 courts on three levels in
Slovakia. The hierarchy consists of 54 district courts, 8 regional courts, Specialized Criminal
Court
2
and the Supreme Court
3
. For each judge serving any year in the judiciary the court (she
serves on) is supposed to publish annual statistical report that provides descriptive data about
1
Evaluation of judges in general, not evaluation of judges on the basis of statistical analysis per se.
2
Specialized Criminal Court is a first instance court for certain serious criminal offences, such as
organized crime or corruption. In the hierarchy it stands on the same level as regional courts although it
differs in its specialization.
3
Constitutional Court is not included as it operates outside of ordinary judiciary.
4
performance of a judges. Of 54 courts the data are available for 51 courts, 3 courts from Prešov
region. Explaining why there are missing data for 3 courts certainly goes beyond the scope of
this paper, even though this fact certainly affects the results d iscussed below. Descriptive data
include information about numbers of incoming and resolved cases, as well as of unresolved
cases and restant
4
cases, information about the number of appellate court decisions regarding
judge's decisions and whether the appellate court affirmed, reversed or annulled these
decisions. All the observations can be easily connected to five main areas - agendas, where
judges decide, these are: criminal, civil, commercial, childcare and administrative agenda. All
agendas, except administrative, can be found throughout the hierarchy for which the first
instance courts are regional courts. However, these reports appear to be rather confusing and do
not provide any context for the data observed (See Appendix). In addition, on the appellate court
levels the reports are filled in a variety of ways, not necessarily having the data in same cell
telling the same story, which makes them at the time unusable, and which forces us to look only
at the district level. Nevertheless, this is not too problematic, district courts have by a huge
margin the largest caseload and the vast majority of cases in the country gets decided exactly
here, so one certainly can justify the importance of putting district courts in the spotlight.
But what do we want to measure when we aim to evaluate performance of individual judges?
Shetreet (2011) claims there are five core values of judicial system: procedural fairness,
efficiency, accessibility, public confidence and judicial independence. Of the five, three can be
ensured on the level of individual case, hence judges can influence them on a day-to-day basis.
Accessibility of judicial system and public confidence are rather systemic features. Certainly
judges do have an impact on them, especially the latter one, but it is an impossible task to
measure their contribution. Judicial independence on the level of a case can be defined as an
absence of decisions that consistently favor actor or a group of actors (Popova 2012) which is
possible to measure, however it cannot be done from the kind of data that are analyzed in this
paper. Procedural fairness refers to dispute resolution that happens "in accordance with fair
procedures" (Shetreet 2011, 5) while it is assumed that these are entrenched in the law and
guaranteed by judicial hierarchy. Lastly, efficiency is valued for a simple reason, as the well-
known legal maxim has it, "delayed justice is justice denied".
2.Measuring quality and efficiency of district court judges
Based on Shetreet's core values of judicial systems we was able to identify two dimensions that
can be found in the available data. What was earlier labeled as "procedural fairness" is for the
4
Restant cases are cases that have been unresolved for more than one year. In the agenda of childcare and
family law the restant cases are cases that has not been resolved for more than 6 months.
5
purpose of this analysis called "quality of a judge". In compliance with "procedural fairness" the
quality judge is a judge that decides on the basis of evidence and in accordance with procedural
rules, reasons his decisions while applying appropriate laws. It is assumed that if decisions of
district court judges do not comply with these criteria it is a reason for reversal or annulment of
the decision by an appellate court. Indeed, regional courts can theoretically suffer from the very
same shortcomings and eventually their decisions can be reversed on the Supreme Court level.
However, in majority of the cases, to be heard on three instances is rare, but in theory it is
possible that a district court judge issues a decision that is reversed on the appellate level, yet
affirmed on the level of the Supreme Court. One can also object that such an assumption has
nothing to do with quality and rather suggests conformity, which certainly is less normatively
biased concept and we do not strongly oppose using it this way. Notwithstanding, we do not find
it too useful to avoid calling it 'quality' as without it the concept raises a considerable concern
regarding the judicial system as a whole, and suggests that hierarchy does not actually serve
intended purposes, but in fact works against them. Defining indicators related to relationship of
a judge with higher court as 'conformity', on the other hand, allows us to ask new questions
regarding judges' positions in the hierarchy and perhaps understand it better. To put it
differently, it can help us address a concern raised by Sajó and Losonci (1993) regarding
bureaucratic nature of judicial positions in post-communist regimes, and to ask whether those
more successful in the hierarchy (such as those in presidency positions), those working there
longer, or those with family ties in judiciary are more likely to agree on particular cases with
appellate courts.
Further, an efficient judge is able to resolve incoming cases, ideally decreases the number of
cases that remained on her docket from the previous period, and is able to prioritize in such a
way that ensures litigants are treated similarly, hence the docket is not filled with cases that
have not been resolved for a long time. If a judge is able to resolve incoming cases, yet among the
unresolved there remains a large share of those that are labeled "restant", probably her ability to
prioritize and treat litigants equally is rather small. Indeed, judge's ability to resolve incoming
cases and consequently decrease number of cases in the docket is largely outside her power and
rather relates to efficiency of the system as a whole. In this research we assume that even though
this is true, there is no reason why judges working in similar conditions, such as at the same
court, should differ from one another. Of course, one may object that cases are not necessarily of
the same difficulty, but once again we assume that judges at the same court, especially working
in similar agenda should not differ from one another; on average the number of incoming cases
per judge in main agendas at the district court has been in 2011-2014 was 389 cases. With
regard to difficulty, judges at different courts certainly may differ, but we will try to control for
6
that in the analysis, but judges at the same court should get roughly cases that are similar in
terms of difficulty.
Indicators of quality
Indicator Name
Note
Q1. Affirmed decisions
Calculated as a share of affirmed decisions from the total number of
decisions of the appellate court.
Q2. Appellation frequency
Calculated as a share total number of decisions from the total number of
decisions issued by a judge. The main problem with the indicator is that
time periods of the two groups are not corresponding as the value of the
former depends on appellate courts, not from a judge herself.
Q3. Reversed or annulled
decisions
Calculated as a share of reversed or annulled decisions from the total
number of decisions issued by a judges. Time periods are not
corresponding.
Indicators of efficiency
Indicator Name
Note
E1. Clearance rate
Calculated as a number of decided cases divided by of all incoming cases
in the time period.
E2. Total number of
unresolved cases
The higher the number is the less able a judge is to focus on new cases.
E3. Total number of restant
cases
The higher the number is the less able a judge is to focus on new cases.
E4. Restant of unresolved
cases
Calculated as a share of restant cases from all unresolved cases. Restant
cases are those that have not been resolved in at least 12 months, except
childcare cases that are considered as restant after 6 months.
E5. Case turnover ratio
Calculated as number of decisions issued by a judge in the last year
compared to all unresolved cases at the end of the year. The indicator
shows judge's ability to resolve unresolved cases were there no incoming
cases.
Table 1. List of indicators
2.1.Data and indicators
5
On the two dimensions, altogether eight indicators were measured; quality consisting of three,
and efficiency consisting of five; all are presented in Table 1. They were developed on the basis
of several interviews with politicians that worked on the creation of annual statistical reports, in
discussions with lawyers and judges. For the analysis the data from 3 years were used (2011-
2013), however they differ with regard to each dimension. Aggregate data from three years were
used to evaluate quality of judges, whereas only the data from the last year were used for the
5
The initial purpose of this measurement was to create a ranking of judges in order to create kind-of
scorecards for each judge that are available at the portal OpenCourts (www.otvorenesudy.sk) operated by
Transparency International Slovakia, the purpose of the data was to provide advocates and general public
with information about individual judges in order for them to adjust their expectations.
7
assessment of judge's efficiency. Altogether, we used data for 943 judges of whom 739 were
coded on all indicators. It needs to be clarified that some of the missing data are missing
intentionally. Data points were coded as missing if there were too few observations in judges'
statistical reports. For instance, if a judge had less than 10 decisions of appellate court with
regard to her decision we chose to rather not evaluate a judge on the particular indicator.
Indeed, these thresholds were chosen arbitrarily and are difficult to justify, but the general
assumption behind them was to exclude observations that could easily skew data and the
subsequent analysis. In the end, only judges for which we have score on every indicator are used
for the analysis. This decisions on one hand does not allow incomplete data about single judge to
distort the analysis, on the other, it may influence the effects of some of the courts, hence
affecting presented results.
2.2. Variables and coding
After calculations of raw data all the score for all indicators was coded on the scale from 0 to 10,
where the better the judge was doing on any indicator the higher the score is. For each observed
value the z-score was calculated; then if the z-score was between -0.2 and 0.2 , hence around the
average, the score was re-coded to 5; if the value was between 0.2 and 0.6 (for indicators where
higher observed values are considered as better) it was re-coded to 6 and accordingly, so if a
judge's z-score on any indicator was above 1.8 the score was re-coded to 10, whereas if the z-
score was below -1.8 it was re-coded as 0. Here we need to admit an important methodological
shortcoming; whereas for 'quality' indicators the average and standard deviation on each
indicator were calculated for specific agendas, the same rule was not applied to 'efficiency'
indicators.
The purpose for the use of z-scores and apply consistent criteria for each indicator was the
assumption that they would create a distribution that would resemble normality. As can be seen
in Figure 1, the assumption did not prove to be right. The reason for that is mainly caused by
some of the 'efficiency' indicators that are heavily burdened on the 0 value, as there are courts in
considerably larger problems than the rest with regard to case load that skews the distribution.
But the problem is not exclusive to 'efficiency' indicators as can be seen in Table 2 reporting
values of scenes and kurtosis for all indicators.
8
Figure 1. Histogram - distribution of the scores, all indicators combined
Indicators of quality and efficiency can always be assigned to a concrete judges as well as the
court where judges serve. In addition to that we were able to control for the agenda a judge
focuses on, gender and whether she has a family in judiciary. The rationale for including the last
of control variables was to see whether family interconnection have any impact on the actual
performance of Slovak judiciary. Generally, Slovak judiciary has often been criticized for
suspected practices of nepotism, and the effect of family proved to be significant for candidates'
success in selection procedures for judgeships(Spáč and Láštic 2014). There were 5 categories in
the agenda variable: criminal, childcare, civil, commercial and "no agenda". A judge was assigned
to specific agenda if at least 50% of her decisions were in this agenda. Category "no agenda"
applied to judges that did not predominantly decide in any particular agenda. The data regarding
family connections were obtained from judge's property declarations as they are obliged to
indicate whether any of their family members works in the judiciary.
Indicator
Q3
E3
E4
E5
Scenes
-0.42
-0.39
0.17
-0.95
Kurtosis
2.46
1.95
2.58
3.38
Table 2. Skewenes and Kurtosis for all indicators
9
3.Analysis
6
For the analysis we used Cumulative Link Mixed Model as the data on the dependent variable
are ordered with 11 factors, while all independent variables are categorical without an order.
Choice of the method enables the analysis to: a) determine whether individual judges play a role
in the observed variance on the dependent variable, b) identify courts with significant effects on
both dimensions, c) identify individual judges and their effects on both dimensions, d) control
for the effect of other variables - such as gender, family connections, agenda, or the effect of
indicators themselves. The analysis will be divided in two parts, as both dimensions were
examined separately. Table 3 presents correlation matrix that suggests some endogeneity
problem on both dimensions - values of Person's r above .500, hence strong correlations, appear
in bold. Originally, we meant to weigh indicators differently against each other as we expected
some endogeneity, but for the purpose of this analysis all indicators are weighted equally.
Q1
Q2
Q3
E1
E2
E3
E4
E5
Q1. Affirmed decisions
1
.036
.523
.007
.056
.040
.074
.034
Q2. Appellation frequency
1
.777
.035
-.121
-.051
.058
.058
Q3. Annulled decisions
1
.039
-.039
.011
.119
.081
E1. Clearance rate
1
.362
.137
-.274
.499
E2. Total - unresolved cases
1
.837
.115
.752
E3. Total - restant cases
1
.517
.680
E4. Restant of unresolved cases
1
.235
E5. Case turnover ratio
1
Table 3. Correlation matrix
I will analyze quality and 'efficiency' indicators independently from each other, and with slightly
different models, and only later compare them and discuss their differences and similarities as
well as implications stemming from these observations for our understanding of how Slovak
judiciary on the lowest level functions. Each analysis will proceed as follows: at first we will
present stepwise modeling checking for the explanatory power of models as compared to more
parsimonious models in order to see whether adding a variable to the model actually improves
it; then we will check for individual effect of variables on the dependent variable. Lastly, we will
present effects of courts and judges given by conditional modes with 95% confidence intervals
based on the conditional variance in order to determine whether courts and judges actually
matter; the results will also be presented with regard to regions in the section concerned with
indicators of quality, and with regard to indicators in the section concerned with 'efficiency'
indicators.
6
The used method was suggested and prepared by Martin Kanovský, Assistant Professor at the Faculty of
Social and Economic Science at Comenius University in Bratislava.
10
3.1.Analyzing 'quality' indicators
For the analysis of 'quality' indicators 2 217 observations were used - 3 for each of 739 judges
included. The model used for the analysis of quality included variables for region, court, judges,
and control for indicators as well as for agenda, and gender and family connections of individual
judges. The rationale for inclusion of 'region' variable is the fact that 'quality' indicators are
centered around relationship between judges and appellate courts, while these appellate courts
are regional courts. There is no specific substantive reason to check for the effect of regions;
perhaps it shall be expected that there should not be any differences among various regions, but
it certainly should be controlled for. Consequently, regions are added to the model as the first
variable, because if there was any effect, it would affect courts within their jurisdiction, and
consequently judges in these courts.
Table 4 presents likelihood ratio tests for cumulative link models for 'quality' indicators. Results
show that all of the indicators, except for family and agenda, have an effect on the observed
performance in terms of quality. They also show that individual judges have an effect on the
performance of judiciary with regard to quality, hence we can reject H0 for the first of the
research questions holding that performance is rather random and judges individually do not
contribute to it.
Model
AIC
LR
Pr(>Chisq)
M0
10 136
M1
region
10 092
45.6
<0.001 ***
M2
region + court
9 935
159.2
<0.001 ***
M3
region + court + judge
9 653
284.1
<0.001 ***
M4
region + court + judge +
indicator
9 643
11.9
<0.001 ***
M5
region + court + judge +
indicator + agenda + gender +
family
9 645
9.8
0.136
Single term deletions
M5a
M5 - (gender)
9 647
4.1
0.043 **
M5b
M5 - (family)
9 644
1.2
0.279
M5c
M5 - (agenda)
9 643
5.7
0.222
Note: p<0.01 *** p<0.05 ** p<0.1 *
N = 2 217 (739 judges)
Table 4. Likelihood ratio tests of cumulative link models - quality
11
Table 5 presents effects and coefficients of all variables included in the most complex model -
M5. However, it is not the best model as the AIC has slightly increased when gender, family and
agenda were added to the model, hence its relative quality has worsened the model. As can be
seen in single term deletions in Table 4, some of this can be attributed to the effect of agenda,
which was taken into account already for the calculation of score on each of 'quality' indicators.
Random Effects
Variance
St.Dev.
region
(Intercept)
0.149
0.387
court
(Intercept)
0.611
0.782
judge
(Intercept)
2.091
1.446
indicator
(Intercept)
0.033
0.181
Coefficients
Estimate
Std.Error
Pr(>|z|)
gender_male
-0.314
0.155
0.043 **
family_yes
0.205
0.189
0.279
agenda_child
-0.358
0.240
0.136
agenda_civil
0.018
0.181
0.919
agenda_comrc
0.148
0.250
0.555
agenda_none
-0.650
0.539
0.228
Note: p<0.01 *** p<0.05 ** p<0.1 *
Table 5. Cumulative Link Mixed Model fitted with Laplace approximation - quality
The only control variable that proved to have a significant effect at 95% confidence level was
gender. Specifically, male judges are more likely to have negative effect on the score in 'quality'
indicators. There is no apparent answer available why something like this can be observed in the
data, but one angle comes to my mind. When meeting with judges, lawyers and politicians in the
process of creating of the indicators of quality, some of the interviewees proposed that perhaps
very low rate of approved decisions on the appellate court suggests corruption. The mechanisms
should work something along these lines: some judges at the district court level are willing to
sign and seal basically any decisions - no matter how irrational - for a financial motivation, as it
secures positive outcome for them, and if the decision gets reversed on the appellate court level,
it gives them an opportunity to wash their hands and decide the case on the basis of
recommendations made by the appellate court decision. If we then connect higher quality with
lower corruption, these results will be in line with other research suggesting that women in
power are less susceptible to corruption than men (e.g. Swarmy, et al. 2001, Esarey and
12
Schwindt-Bayer 2015) due to, for instance, higher risk aversion by women (Esarey and Chirillo
2013).
Positive effect
Negative effect
Judges
TOP 10 (49 judges):
Šenkár (ZA), Lukáč (LM), Hartelová (NO), Klimková
(HE), Malatká (ZA), Marková (RS), Dzúriková (ZV) ,
Koreň (RV), Gandelová (BA II), Mitterpák (KE II)
TOP 10 (57 judges):
Šulák (KN), Melníková (VT), Rybárik (ZA), Kollár
(ZV), Ilgová (MT), Lopuch (MI), Dobrík (MT),
Hozáková (CA), Šuťák (PO), Vaľuš (VT)
Courts
Bratislava V, Prievidza, Rimavská Sobota, Trebišov
Humenné, Košice II, Námestovo, Topoľčany
Table 6. Judges and courts with either positive or negative effect with 95% CI - quality
Based on the analysis it is possible to identify individual effect of regions, court and judges on
the dependent variable. List identifying those with effects significant at 95% confidence level are
presented in Table 6. The model used for these results excludes gender and family variables, as
they are redundant when the aim is to identify individual judges' effects on the overall
performance. In such circumstances it does not make sense to control for the effect of these
attributes as they, generally speaking, cannot change across time and are independent from the
court a judge works at or the agenda in which she decides majority of cases.
Figure 2: Region effects given by conditional modes with 95% confidence interval based on the
conditional variance - quality
-1
-0,5
0
0,5
1
1,5
BA
ZA
PO
TT
NR
BB
KE
TN
Region effect
Region
13
Individual effects can be also plotted given by conditional modes with 95% confidence interval
based on the conditional variance. These are presented in Figure 3 for 'region' variable, Figure 4
for courts, and Figure 5 for judges. While none of the regions proved to have any significant
effect, distributions of courts' and judges' effects are much more diverse. Among the regions only
Trenčín region gets very close to positive effect, among courts there are four courts with positive
and four courts with negative effect, and additionally there is around 50 judges on each side of
the plot suggesting that almost every seventh judge significantly influences the overall
performance of judiciary in terms of quality.
Figure 3: Court effects given by conditional modes with 95% confidence interval based on the
conditional variance - quality
All in all, what the analysis of performance of judges in terms of quality showed is that judges do
matter and they seem to individually contribute to how the judiciary functions in this regard.
Also, results indicate male judges are more likely to have negative effect on the overall quality of
judiciary. Then, if we then accept that women are less likely to get involved in corruption, it may
support a hypothesis raised in the interviews with judges, lawyers and politicians that low
quality, measured as a rate of decisions approved by a higher court, suggests higher risk of
corruption. Also, the analysis enabled us to identify judges' and courts' individual contribution to
the overall performance. Out of 51 examined courts, eight have some effect at 95 % confidence
level, which is 15.7 % of all courts, and 106 out of 739 judges have either positive or negative
effect, accounting for altogether 14.3 % of all judges.
-3
-2,5
-2
-1,5
-1
-0,5
0
0,5
1
1,5
2
2,5
Court effect
Court
14
Figure 4: Judge effects given by conditional modes with 95% confidence interval based on the
conditional variance - quality
3.2.Analyzing 'efficiency' indicators
The analysis of efficiency included 3 695 observations. Unlike the model used in the previous
section, for the analysis of efficiency we chose not to include 'region' variable. As a matter of fact,
when cumulative link mixed model analysis was applied to model with this variable, it seemed to
have an effect, however there is no substantive reason for that and that is a reason why we chose
not to report it. Perhaps, the choice we made is not justified sufficiently and there exists a reason
why such a variable should be included. Possibly, the cases that come to courts differ between
the regions, or there is a problem with capacity of district courts caused by their affiliation with
region, but we do not have any knowledge that could clarify this connection, as the number of
judges for each court is supposedly decided regardless of the region in which they operate and
only on the basis of needs of particular courts.
-6
-4
-2
0
2
4
6
Judge effect
Judge
15
Model
AIC
LR
Pr(>Chisq)
M0
16 856
M1
court
16 494
364.71
<0.001 ***
M2
court + judge
15 941
554.81
<0.001 ***
M3
court + judge + indicator
15 862
80.454
<0.001 ***
M4
court + judge + indicator +
agenda + gender + family
15 414
460.73
<0.001 ***
Single term deletions
M4a
M4 - (gender)
15 412
0.42
0.518
M4b
M4 - (family)
15 414
2.79
0.095 *
M4c
M4 - (agenda)
15 838
432.27
<0.001 ***
Note: p<0.01 *** p<0.05 ** p<0.1 *
N = 3 695 (739 judges)
Table 7. Likelihood ratio tests of cumulative link models - efficiency
Cumulative link mixed model analysis shows that courts, judges, indicators and agenda all
significantly improve the model explaining performance in terms of efficiency. At the 90 %
confidence level also family variable would have significant effect, but even though it is reported
in results the confidence level is not sufficient (Table 7). Likelihood ratio statistics and the
decrease on AIC suggests that individual judges as a matter of fact has the largest impact on the
strength of the model, followed by the agenda variable.
Random Effects
Variance
St.Dev.
court
(Intercept)
0.713
0.845
judge
(Intercept)
0.643
0.802
indicator
(Intercept)
0.087
0.294
Coefficients
Estimate
Std.Error
Pr(>|z|)
gender_male
0.064
0.099
0.518
family_yes
0.200
0.120
0.095 *
agenda_child
-0.915
0.154
<0.001 ***
agenda_civil
-2.617
0.124
<0.001 ***
agenda_comrc
-1.906
0.163
<0.001 ***
agenda_none
-1.266
0.359
<0.001 ***
Note: p<0.01 *** p<0.05 ** p<0.1 *
Table 8. Cumulative Link Mixed Model fitted with Laplace approximation - efficiency
16
Table 8 shows that all agendas have negative effect on the performance in 'efficiency' indicators
as compared to criminal agenda. Substantively, the observed results seem sensible. Criminal
agenda is usually considered a priority in judicial systems around the globe as it directly deals
with human rights and hence creates incentives for countries and their judicial system to make
criminal agenda their priority. Coefficients estimates suggest that the least negative effect of the
remaining agendas have cases related to childcare which similarly can be substantively
explained. On the other hand, a change on the agenda variable from criminal agenda to civil or
commercial agenda decreases judges' scores on 'efficiency' indicators by 2.6 or 1.9 respectively
suggesting that courts' capacities in these agendas are considerably inadequate. This issue
deserves a particular attention as inability of a judicial system to provide efficient access to
justice in civil and commercial cases is believed to compromise economic growth (e.g. Pinheiro
1996, Olson 1996).
Positive effect
Negative effect
Judges
14 judges:
Králová (BA IV), Majerčík (KN), Gazdag (KE II),
Kotus (KE II), Jamrišková (MT), Káčeriková (MT),
Macúchová (NM), Miklová (NZ), Križanová (NZ),
Krajčírová (PN), Dusza (RS), Boros (RS), Gallo (RV),
Koreň (RV)
15 judges:
Gandelová (BA II), Redenkovičová (BA IV),
Žideková (CA), Schweitzerová (GA), Novotná
Mliárcsik (KE I), Frankovič (KE II), Podhorcová
(LM), Balážová (LC), Záhorák (NZ), Tvrdá (PN),
Sikorjak (PO), Tengely (RV), Vargová (VT), Ševčík
(ZH), Lečko (ZA)
Courts
11 courts:
Bánovce n/Bebravou, Banská Bystrica, Dolný
Kubín, Dunajská Streda, Humenné, Lučenec, Nitra,
Považská Bystrica, Ružomberok, Senica, Veľký
Krtíš
9 courts:
Bratislava I, Bratislava V, Košice II, Levice, Malacky,
Partizánske, Pezinok, Piešťany, Žilina
Indicators
E5.Case turnover ration
E1.Clearance rate
E4.Restant of unresolved cases
Table 9. Judges, courts and indicators with either positive or negative effect with 95% CI -
efficiency
Once again, the analysis enables to estimate the effect individual judges, courts and indicators
have on the observed performance; those with effect significant at 95% confidence level can be
found in Table 9. The model, just as in the previous section, does not account for the effect of
gender and family variables. As compared to the model looking at 'quality' indicators there is
much less judges with significant effects, whereas the number of courts increased more than
17
twice. Unlike in the previous analysis, 3 out of 5 indicators have significant effects, suggesting on
one hand the need to address the lack of normality in distributions on particular indicators, and
but on the other it allows for completely new perspectives in the interpretation of data.
Figure 5: Court effects given by conditional modes with 95% confidence interval based on the
conditional variance - efficiency
When individual effects are plotted one particular thing is striking. Figure 5 shows effect of
courts by conditional modes with 95% confidence intervals, and what can be seen is that there is
much larger difference between courts that are performing best to those that are performing
worst. Whereas in 'quality' indicators those performing significantly better or worse were
located close to 0 value, with regard to efficiency as much as three court have effect of at least -1
and comparable effects can be observed among the courts with significantly positive effect.
Figure 6 on the other hand suggests the effect of individual judges on the efficiency is weaker
than the effect that was shown with regard to 'quality' indicators. Generally, the observed
variance appears much smaller, hence one can argue that efficiency is mostly a matter of the
management of judicial system and depends more on capacities of courts than on individual
judges working in the system. In addition, Figure 7 shows different effects indicators have on the
overall results. The reason why the effect of indicators of efficiency informs the analysis is the
following. To simplify, significantly positive effect suggests the average observed in the original
descriptive data on any particular indicator is lower than the median, hence skewing the
distribution in favor of higher scores. Then, if 'E5. Case turnover ratio' has significantly positive
-3
-2
-1
0
1
2
3
Court effect
Court
18
effect it indicates that there are judges and courts whose ability to resolve previously unresolved
cases is much lower than it is for majority of judges and courts. It may signify existence of few
courts with insufficient capacities to overcome problems that they have not resolved in the past.
Contrarily, the negative effect of 'E1. Clearance Rate' indicator shows there are few judges or
courts who were able to resolve incoming cases while majority of courts did not have adequate
resources and capabilities.
Figure 6: Judge effects given by conditional modes with 95% confidence interval based on the
conditional variance - efficiency
Figure 7: Indicator effects given by conditional modes with 95% confidence interval based on the
conditional variance - efficiency
-4
-3
-2
-1
0
1
2
3
4
Judge effect
Judge
-0,6
-0,4
-0,2
0
0,2
0,4
0,6
0,8
E4
E1
E2
E3
E5
Indicator effect
Indicator
19
To answer the research questions asked in this paper, the analysis showed judges do play a role
in the overall efficiency of courts, however contrary to quality the efficiency is mostly a matter of
the management of judicial systems. Judges matter, yet only 29 of them - 3.9% has significant
positive or negative effect on the scores on the dependent variable. Contrarily, there is 20 out of
51 courts, hence 39.2% with statistically significant effect on the performance. In addition,
especially cases in civil and commercial agendas seem to be resolved much less efficiently than,
for instance, criminal cases, and although some of this difference can be explained and even
justified, the inability of the system to effectively deal with these cases may threaten economic
growth. And finally, the effect particular indicators have on the overall performance suggests
there are few courts in considerable problems, perhaps inherited from the past, or not
sufficiently dealt with rather recently, while on the other hand there are few courts capable to
satisfactory resolve incoming cases, yet the results suggest those are rather exceptions from the
rule.
4.Discussion
The analysis shows that individual judges do play a role in the overall performance of the
judiciary and individually contribute to quality and efficiency of the system, and consequently to
satisfaction of general public with the judiciary and institution's trustworthiness. However, the
effect of judges is not the same in the two dimensions. Whereas for the indicators of quality
14.3% of judges has either positive or negative effect, for 'efficiency' indicators it is only 3.9%.
Contrarily, more courts have significant effect on the latter dimension - 39.2% of courts have
statistically significant effect on the overall performance of the judiciary in terms of efficiency
compared to 15.7% of 51 examined courts with effect on 'quality' indicators. Based on that , it
can be argued that efficiency is to a larger extent in the hands of those responsible for
management and administration of the judicial system - in Slovakia it is the Ministry of Justice
and Judicial Council. Further, with regard to quality, gender variable proved to be significant.
Male judges contribute to the overall performance on the indicators of quality negatively. There
is no apparent explanation for this at hand, but it seems plausible to connect low quality with
higher corruption risks. Efficiency, on the other hand, is to a large extent influenced by agenda.
Judges working in criminal agenda seem to be much more effective than judges in any other
agenda. Exceptionally bad is the efficiency of judges working in civil and commercial agendas
which may endanger Slovakia's ability to secure economic growth.
Of 739 Slovak district court judges that were included in the analysis 127 have some statistically
signficant effect in at least one of the dimension, which accounts for 17.2% of the sample. Eight
20
of these have some significant effect on both of the dimensions: four contribute positively in
terms of quality as well as efficiency, two have differentiated effect, and two have negative
individual effect on both dimensions. Based on that it is necessary for judicial administration,
political power, media, non-governmental organizations, the general public as well as academia
to pay attention to judges and their professional careers, even in ordinary judiciaries - from their
recruitment to the judiciary, through their time in the hierarchy, their promotions to
administrative positions, such as to the positions of court presidents or promotions to higher
courts, to their eventual removal from the office.
References
Bojarski, Lukasz, and Werner Stemker Köster. The Slovak judiciary: its current state and
challenges. Bratislava: Open Society Foundation, 2012.
CEPEJ. "Report on European judicial systems - Edition 2014 (2012 data): efficiency and quality
of justice." 2014.
Cross, Frank B., and Stefanie Lindquist. "Judging the Judges." Duke Law Journal 58, 2009: 1383-
1437.
Esarey, Justin, and Gina Chirillo. "'Fairer Sex' or Purity Myth? Corruption, Gender, and
Institutional Context." Politics and Gender 9(4), 2013: 390-413.
Esarey, Justin, and Leslie Schwindt-Bayer. "Women's Representation, Accountability, and
Corruption in Democracies." Sumbitted to ECPR Standing Group at ECPG Conference at University
of Uppsala, 11-13 June 2015. 2015. http://ecpr.eu/Filestore/PaperProposal/ba408c61-5bcd-
4372-b7be-0fabe50ae23a.pdf (accessed Oct 22, 2015).
Flanders, Steven. "Evaluating judges: how should the bar do it?" Judicature 61(7), 1978: 304-310.
Choi, Stephen, and Mitu Gulati. "A Tournament of Judges?" California Law Review, Vol. 92, No. 1,
2004: 299-322.
Krastev, Ivan. "The transparency delusion." Eurozine. Feb 2, 2013.
http://www.eurozine.com/articles/2013-02-01-krastev-en.html (accessed Oct 19, 2015).
Olson, Mancur. "Distinguished lecture on economics in government: big bills left on the
sidewalk: why some nations are rich, and others poor." Journal of Economic Perspectives, vol. 10,
no. 2, 1996: 3-24.
Pinheiro, Armando Castelar. Judicial System Performance and Economic Development. 1996.
http://www.bndes.gov.br/SiteBNDES/export/sites/default/bndes_pt/Galerias/Arquivos/conhe
cimento/ensaio/ensaio2.pdf (accessed Oct 22, 2015).
21
Popova, Maria. Politicized Justice in Emerging Democracies: A study of Courts in Russia and
Ukraine. New York: Cambridge University Press, 2012.
Sajó, András, and Vera Losonci. "Rule by Law in East Central Europe: Is the Emperor's New Suit a
Straightjacket?" In Constitutionalism and Democracy: Transitions in the Contemporary World,
edited by D. Greenberg, 321-338. Oxford: Oxford University Press, 1993.
Shetreet, Shimon. "Judicial independence and accountability: core values in liberal democracies."
In Judiciaries in Comparative Perspective, edited by H.P. Lee, 3-24. Cambridge: Cambridge
University Press, 2011.
Schwab, Klaus, and Xavier Sala-i-Martin. The Global Competitiveness Report 2013-2014. World
Economic Forum, 2013.
Spáč, Samuel, and Erik Láštic. "The Untouchables: The Politics of Judicial Selection and Turnover
in Slovakia between 1989 and 2013." presented at ECPR Joint Sessions 2014 in Salamanca. 2014.
Sterling, Joyce, E.Keith Stott, and Steven Weller. "What judges think of performance evaluation: a
report on Colorado survey." Judicature 64(9), 1981: 414-424.
Swamy, Anand, Stephen Knack, Young Lee, and Omar Azfar. "Gender and Corruption." Journal of
Development Economics 64(1), 2001: 25-55.
TASR. "Súdy majú katastrofálnu bilanciu. Nedôveruje im skoro 70% Slovákov [Courts have
terrible balance. Almost 70% of Slovaks do not trust them]." June 20, 2012.
http://spravy.pravda.sk/domace/clanok/247416-sudy-maju-katastrofalnu-bilanciu-
nedoveruje-im-skoro-70-slovakov/ (accessed May 5, 2015).
Transparency International. "Global Corruption Barometer." 2013.
22
Appendix
... Data from 4,333 annual statistical reports of 1,012 district court judges from 2011 to 2015 were analyzed. Reports contain descriptive information about a judge's performance in a given year -the number of assigned cases, decided cases, resolved 19 For full version of the research see (Spáč, 2015) cases and unresolved cases at the end of the period, as well as information on how appellate courts decided on judges' decisions. Data in these reports, however rich in value, actually tell us very little about one judge's performance without placing them into comparative context. ...
Article
Full-text available
In 2012, a comprehensive judiciary reform came to effect in Slovakia, which aimed (among other things) to increase the transparency of the judiciary. To this end, the law has mandated that all the court decisions must be published online, selection procedures for judges be open to public oversight, and judges have to annually declare their family ties within the judiciary as well as performance statistics. This study looks into the impact the transparency reform had on the current state of the Slovak judiciary. A consensus among key stakeholders was found that making judiciary more transparent was worthwhile. However, it is not too clear to what extent more public accountability improved the quality and integrity of the Slovak judiciary so far.
Technical Report
Full-text available
The authors of this report were asked “to conduct an initial mapping and make recommendations to address the problem of judicial transparency and accountability in Slovakia”. The emphasis of the report should be “the recommendations to the Government and the non-governmental sector to improve transparency and accountability in accordance with international standards and trends and with regard to Slovak government plans, and to decrease or eliminate sources of political influence upon the system”. From Conclusions The Slovak judiciary is in disarray and in turmoil. It is in disarray because there is hardly any country in the EU where public confidence in the judiciary is as low as in Slovakia. This in itself is a reason for concern, as a judiciary that has no public confidence loses its legitimacy. Additionally, the actions of the judiciary itself, especially the mass claim for wage discrimination, are not very helpful in restoring public confidence. The public generally does not have confidence in the impartiality of its judges, believes the judiciary is corrupt and complains about the lengthiness of legal procedures. Allegations concerning partial judges and judicial corruption are many, but only few can be corroborated with hard data. Strangely, not many investigations or in-depth (parliamentarian) inquiry has ever been conducted into these allegations. [...] The Slovak judiciary is also in turmoil. The clear impression of our visit to Slovakia in March 2011 and of all the documentation we studied before and after our visit is that the Slovak judiciary is as politicised as Slovak society is as a whole. There is a public debate going on about the independence of the judiciary between the president of the SNCJ on the one side and members of the Government on the other. The debate is conducted with a fierceness that in many other countries would be regarded as undignified for any representative of the judiciary. One may argue that the way the debate is conducted in itself is in conflict with internationally accepted standards. The appointment in 2006 of Mr Harabin as Minister of Justice under the previous government and – in 2009 – his election to the post of president of the Supreme Court (and therefore also president of the Slovak National Council of the Judiciary), have sparked a long-lasting conflict between the members of the judiciary. From the interviews we held during our visit to Slovakia, we have gotten the impression that within the judiciary roughly 10% of the judges support Mr Harabin, while another roughly 10% oppose him...
Article
At the turn of the twenty-first century, an important pair of studies established that greater female representation in government is associated with lower levels of perceived corruption in that government. But recent research finds that this relationship is not universal and questions why it exists. This article presents a new theory explaining why women’s representation is only sometimes related to lower corruption levels and provides evidence in support of that theory. The study finds that the women’s representation–corruption link is strongest when the risk of corruption being detected and punished by voters is high – in other words, when officials can be held electorally accountable. Two primary mechanisms underlie this theory: prior evidence shows that (1) women are more risk-averse than men and (2) voters hold women to a higher standard at the polls. This suggests that gender differences in corrupt behavior are proportional to the strength of electoral accountability. Consequently, the hypotheses predict that the empirical relationship between greater women’s representation and lower perceived corruption will be strongest in democracies with high electoral accountability, specifically: (1) where corruption is not the norm, (2) where press freedom is respected, (3) in parliamentary systems and (4) under personalistic electoral rules. The article presents observational evidence that electoral accountability moderates the link between women’s representation and corruption in a time-series, cross-sectional dataset of seventy-six democratic-leaning countries.
Chapter
The study of judicial independence is important in national legal systems as it is an essential guarantee for democracy and liberty. Judicial independence is also an essential feature in ensuring a globalised economy. Corporations must have confidence in the impartiality and independence of the tribunals that will adjudicate disputes in the multiple jurisdictions in which they operate around the world. Recent decades have witnessed a marked increase in the relative role of the judiciary in society. This general trend is shared by countries with different legal traditions and various systems of government. The judiciary is a significant social institution, and like the other branches of government, contributes to shaping the life of the community. The increasing role which the judiciary has assumed warrants some re-examination of the conceptual framework and the theoretical rationales which define its position in relation to the other branches of the government. One of the most significant aspects of the role of the judiciary in society is its independence and impartiality.
Book
Why are independent courts rarely found in emerging democracies? This book moves beyond familiar obstacles, such as an inhospitable legal legacy and formal institutions that expose judges to political pressure. It proposes a strategic pressure theory, which claims that in emerging democracies, political competition eggs on rather than restrains power-hungry politicians. Incumbents who are losing their grip on power try to use the courts to hang on, which leads to the politicization of justice. The analysis uses four original datasets, containing 1,000 decisions by Russian and Ukrainian lower courts from 1998 to 2004. The main finding is that justice is politicized in both countries, but in the more competitive regime (Ukraine) incumbents leaned more forcefully on the courts and obtained more favorable rulings.
Article
We suggest a Tournament of Judges where the reward to the winner is elevation to the Supreme Court. Politics (and ideology) surely has a role to play in the selection of justices. However, the present level of partisan bickering has resulted in delays in judicial appointments as well as undermined the public's confidence in the objectivity of justices selected through such a partisan process. More significantly, much of the politicking is not transparent, often obscured with statements on a particular candidate's "merit" - casting a taint on all those who make their way through the judicial nomination process. We argue that the benefits from introducing more (and objective) competition among judges are potentially significant and the likely damage to judicial independence negligible. Among the criteria that could be used are opinion publication rates, citations of opinions by other courts, citations by the Supreme Court, citations by academics, dissent rates, speed of disposition of cases, reversal rates by en banc panels and the Court, and so on. Where political motivations drive the selection of an alternative candidate, our proposed system of objective criteria will make it more likely that such motivations are made transparent to the public. Just as important, a judicial tournament for selection to the Supreme Court will serve not only to select effective justices, but also to provide incentives to existing judges to exert effort.
Article
Using several independent data sets, we investigate the relationship between gender and corruption. We show using micro-data that women are less involved in bribery, and are less likely to condone bribe-taking. Cross-country data show that corruption is less severe where women hold a larger share of parliamentary seats and senior positions in the government bureaucracy, and comprise a larger share of the labor force. (C) 2001 Elsevier Science B.V. All rights reserved. JEL classification: K42, J16.