The practice of the usk and the scientifically questionable results of the kfn studies
In our article killer game alarm in germany. The difficult business of age classification and the longing for unambiguity we tried to bring the discussion about "killer games" to objectify and to bring in aspects, which were omitted so far. In doing so, we dealt in detail with the kfn study "media consumption, school performance and youth violence". In this new article, we address the kfn staff’s response to our previous article, as well as another kfn study. In addition, we present some important aspects of the reviewers’ practice that were not taken into account in the studies.
We summarize the results of our argumentation in five points for readers in a hurry:
- The kfn study media consumption, school performance, and youth violence argues that playing violent computer games is causally related to academic failure and real-world violence among young people. The thesis was uncritically taken up by the media and spread as a scientifically confirmed fact. The thesis is not supported in the study. Rather, correlations are reinterpreted as causal relationships. Our criticism in this regard in our first article could not be dispelled by the kfn authors’ reply (transparent gross delusion).
- The kfn study "age classification of computer games by the usk "1 raises accusations that the usk’s age ratings are too lax. This is not substantiated. The claim to objectify age ratings also stops at trying to play gamesdescriptions to objectify. The actual decision-making process is carried out by a single kfn tester. The decision process is not documented in the study. The reference to various violent elements in a game cannot justify why the game could influence a certain age group, but not another one. Thus, kfn’s ratings remain at the level of an opinion deception. There would be nothing wrong with that, if they were not provided with the claim of a scientific result.
- Instead of detailed case studies, the general and scientifically questionable argument that the active role of the player in violent games has more problematic consequences than the passive viewing of a comparable film, has to be used to tighten the age rating. In our previous article, we argued that gamers are generally more detached from the game than viewers are from the film, and we gave a detailed reasoning for this. Recent studies such as those of the british board of film classification (bbfc) and the hamburg hans bredow institute come to the same conclusion.
- The research situation remains ambiguous and the legal criteria are not operationalized (nor can they be operationalized, cf. Hans bredow institute s. 102). (5) the practice of the usk tries to correspond to this situation by putting the emphasis of the decision on the careful interpretation and weighing of the risks of each individual case. In doing so, it can rely on review panels that are always composed differently. The individual usk reviewers decide independently – both of the industry and of the state youth authorities.
- Problems of transparency at the presentation of the decisions of the usk are available. These are mainly due to the difficulty of adequately depicting complex processes of discussing and weighing decisions. Nevertheless, this state of affairs is unsatisfactory. Criteria and guidelines for plausibility and transparency in the presentation of decisions are continuously being developed in the usk. Since 2006, there has been a separate working group for this purpose.
Limits of empirical research and the search for unambiguous results of impact research
In a detailed response to our article, the authors of the kfn study acknowledged our objection as justified and confirmed the relativity of their results. This is in pleasant contrast to the excited media discussion, even if the authors tried to justify their approach by the fact that they had been confirmed by other scientific papers. They referred to a number of studies in which they believed to find evidence for their theses. However, they only used the studies that supported their own theses and interpreted them in a one-sided way. A procedure that does not exactly adorn scientists.
A new study by the hans bredow institute in hamburg clearly establishes the lack of research results that are effective in practice:
The research situation with regard to the question of the effect of violence in screen-based games – just like the effect research on television programs containing violence – is still ambiguous. In the public discussion, the findings of individual studies are often overinterpreted, generalized inappropriately, statistical relationships are reinterpreted as causal relationships, or generalized to other media offerings.
Bredow study, p. 66
Hard words to put behind the mirror at kfn.
We related the last two objections from bredow to the kfn studies in our article and gave detailed reasons for them. We do not see that the rebuttal invalidates them. As experts we have been looking for arguments in the empirical research of effects for our work for years. Unfortunately so far with very modest success. Strictly speaking, this is not surprising, because the effects of computer gaming are a highly complex matter and are related to a multitude of conditions that empirical research can (necessarily) only do short justice to. Since empirical results cannot reflect the complexity of the real world, i.E. Have only limited significance, non-statistical studies such as those conducted for many years by the university of applied sciences in koln – which empirical researchers naturally do not like to hear – are all the more important and productive.
We are also displeased that the director of the kfn still uses the studies of his institute, although they contribute relatively little to the topic, in order to serve existing uneasiness and prejudices in a frictionless way. We did not hear that the staff of the kfn had publicly distanced themselves from this because their "scientific conscience" was beating.
In a new study, kfn staff attempted to prove their previously proclaimed thesis that the usk’s age ratings are downright irresponsibly lax. They developed a set of criteria that they used to assign their own age ratings to 62 games that had been labeled by usk/oljb. In the majority of cases, they came up with higher rankings.
To clarify: like the kfn staff, we ame that violent computer games can influence or endanger young people of different ages – even if such effects have not yet been proven beyond doubt by the scientific community. Otherwise we would not have to work for the usk as honorary experts.
In question is not, whether, but such as the institutions for the protection of minors assess the potential danger in concrete individual cases, how and with which procedures sensible decisions about age ratings can be made. Since the procedures of the usk and the procedures proposed by the kfn are indeed quite different, we would like to present here some central points of the procedure that have so far received little or no public attention.
Open processes or bearer transparency through standardization?
Decisions about age ratings are complex processes of case-by-case assessment that cannot be standardized:
"At the legal level, therefore, no explicit criteria are set, nor are any standards defined, on the basis of which concretization by the organizations appointed for this purpose is to take place. … First, it should be noted that this openness allows for a certain flexibility that can be functional for the protection of minors, since both media content and the findings and evaluations that form the basis for judging whether a medium has developmentally impairing potential are subject to rapid change.
The criteria are more concrete – again in accordance with the high risk to legal interests – but also contain a large number of terms that are open to interpretation (‘glorifying’, ‘people’, ‘in a way that violates human dignity’, ‘obviously suitable’). The criteria are disputed in detail.
Bredow study, s. 102
In this situation – ambiguity of empirical research, no clear, concretizable legal criteria – the decision about age ratings can, in our opinion, only be optimized by means of procedural regulations in which the discussion and interpretation of the criteria for each individual game plays a central role, and not by means of standardizations of content.
We consider two points to be central here:
1. Decision-making through deliberation of always different bodies
In our experience as usk reviewers, the quality of the decisions is determined by the fact that the deliberations and discussions of the review panels are dynamic and lively in the sense that new points of view are always included in the discussion and rigid schemes are avoided. We consider it a strong point of the usk’s concept to be able to rely on 55 experts who work on a voluntary basis. As a result of this relatively high number, it is virtually impossible for two equally constituted boards to meet. Routine and schematic approaches can be hard to creep into. In fact, dealing with people who are all committed to the protection of minors, but who have different backgrounds, are men and women of different ages, and come from different states, means that we are constantly confronted with new views and insights. This forces us to reconsider and question our own positions.
Often discussions about the trial decisions are controversial and end with majority decisions. This is by no means convenient for us reviewers, and the danger of possible inconsistency cannot always be ruled out. Nevertheless, we consider the openness of the process to be a strength of the usk decisions and regret that this feature of the deliberative process was not mentioned, analyzed, or evaluated in any of the studies.
The ies are complex, and the hope that above all differentiated catalogs of criteria will create unambiguity, transparency and bindingness remains a pious wish. The problem is that each test criterion must be seen and evaluated in the context of the whole game. Humorous scenes in a violent game can seem like cynical comments and indicate a risk of indexing for the game. In another game, formally similar components can relativize the seriousness of the gameplay and strengthen the player’s distance. Even both interpretations can be possible and comprehensible for one and the same game – a complex discursive balancing act in the argumentation, which cannot be operationalized.
Such discussions are not only the real core in the age rating process – they also make our work as reviewers interesting and creative. Well-structured criteria catalogs – if they remain changeable, i.E. If they are regularly checked for their usefulness – could help, but nothing more.
2. Independence of the experts
It is also of central importance for us that the independence of the experts continues to be guaranteed. This means, on the one hand, that the assessors act independently of their employers, and on the other hand, that neither the industry nor the state youth authorities can influence their decisions. Whether the regional youth authorities will accept the decision of the committee is another question.
There has been no industry influence so far. It is true that the work of the usk is financed by fees paid by the industry. But the age rating is required by law and the fees are fixed, so it is completely independent of the result.
With the procedure practiced by the usk – as with any other procedure – it cannot be ruled out that problematic decisions will be made in individual cases. In such a case, only the procedure – in the usk the possibility of the representative of the supreme state youth authority as well as the applicant to appeal – can also mitigate the risk. Whether other persons or institutions should be involved would be a decision to be made politically.
Notes on the decision-making procedure in the kfn study
The authors of the kfn study preferred a completely different procedure to arrive at an age rating. The basis of their decision is the report of a tester who has played through the game completely. The usk has an extensive criteria grid at its disposal to prepare a detailed description of the game. For the criteria that are considered particularly significant, he records scenes from the play and attaches them to the report, in which he also gives the age rating. Two scientific staff members of kfn check the report for consistency of content and plausibility. Afterwards, there is a 30-minute discussion with the tester (final report s. 22).
Although the kfn staff states that "the central legal concepts for the questions of age classification or ageing of children are not yet clear, they are still not clear. Indexing … Undefined legal terms" (final report s. 26), the kfn believes that it can do justice to the protection of minors with a catalog of criteria as the central instrument for age rating.
Comparison of procedures
If you compare the procedures of the usk and the kfn, you will notice one thing in common: in both cases, the game is played through completely by a tester, who then writes a report. The reports of the kfn testers (ca. 25 pages, but almost half of the text is taken up by the criteria question and matching yes/no boxes, s. 80 ff.) are significantly longer and more standardized than those of the usk testers. In terms of content, however, the same focal points are dealt with to a large extent, whereby the usk testers restrict themselves to the aspects that actually occur in the respective game.
The report (including the recorded game scenes) of the kfn tester, however, already contains the core of the decision-making process, while the tester’s report in the usk panel is only a prerequisite for the actual decision-making process. That is: at kfn, the decision-making process – except for a plausibility check – is already completed with the test report, while the usk reviewers only get a first overview of the game when they read the test report. After that the game is presented in the usk review session. No recorded game scenes are shown, but the game is played live. Of course not the whole game from start to finish, but usually the first level first. Then follow saved scores of other passages, sequences from different levels, so that the usk panel can get an overview of the whole game, not only about occurring violent peaks.
This method differs qualitatively significantly from the kfn. Recorded sequences of the most violent scenes of a game often have a "trailer effect". Everyone knows this from the cinema, when the three most gripping scenes of a movie are shown in the previews. Some people are later disappointed when they see the whole movie. Similar is the case with the selected recordings of the kfn team. However, the presentation of the game offers the usk committees the opportunity to gain an overall impression of the game and to be able to classify problematic content in this overall context.
The second difference is that the evaluators do not just passively consume a presentation, but intervene: for example, by asking questions such as "what happens if…? I would like to see again how…"In addition, the evaluators themselves can switch from the role of spectator to that of active player in order to get an impression from this perspective as well. So there is no closed prasentation predetermined by the tester.
That the testers deliberately withhold problematic scenes from the committees – a malicious interest for which there is no evidence – cannot be completely ruled out, as was the case, incidentally, with the kfn testers. But it cost them their beloved job. Game over.
After the review of the game follows the discussion of the age rating. It is often a controversial ie. Sometimes reviewers are persuaded by the arguments of their colleagues and change their positions so that a unanimous result is reached. Sometimes, even after a long discussion, no agreement is reached, and the decision is z.B. Made with 3:2 votes. But there is one thing that every reviewer knows from his or her experience of arguing, weighing and interpreting the meaning of the respective criteria: that there are usually good reasons for one decision as well as for the other. The idea that the opinion deviating from their own assessment is a "wrong decision" no one came to the conclusion.
The kfn employees are different. Although they state in their reply to our article that there is subjective leeway in the game rating, they define deviating ratings of the usk committees as a "mistake" "misjudgments", consider their own opinions to be objectively correct. This should not happen to social scientists.
Possibly this faux pas is due to the fact that the same homogeneous group always works together in kfn. Only in a rigid group whose members confirm each other can the illusion arise that they are the only ones to make "correct" decisions. Because of this illusion, the kfn employees probably cannot understand that the usk reviewers, whose integrity they do not doubt, come to different conclusions than they do after responsible and differentiated discussion. Therefore, they concoct an abstruse theory of conspiracy, according to which well-intentioned, but slightly dim-witted review boards allow themselves to be manipulated by hardened testers.
The central argument for the kfn staff’s preference for higher age ratings is: due to the interactivity, i.E. The active role of the player, more lasting consequences are to be expected from the consumption of games containing violence than from the consumption of comparable films. The authors of the studies use formulations such as "the gambler is actively involved in the fight against poverty "others kill and torture". (summary of the research report, s.1).
The premise is scientifically questionable. This is the result of the hans bredow institute after reviewing the relevant studies:
Empirical research results show that the involvement in games is generally higher, but due to the empathetic co-experience of events, which is essential in the reception of films and which is assigned a certain character of reality even in the case of clearly fictional material, the same visual representations in a film seem more threatening than in screen games, which always remain tied to the player’s own actions. Despite the flow experience, players are usually aware that they are playing a game, which gives them a certain distance to the content.
Final report s. 139
While in this country a tightening of the age rating is demanded, in england the possibility of a relaxation is currently being discussed. The bbfc, which is responsible for the age rating of films and problematic computer games in england, concludes after a study of the behavior and experience of computer gamers that the interactivity strongly ties the players to the game, but that the focus is on the achievement of the game goal, not on the viewing of violent scenes. The latter is a viewpoint that tends to be held by passive spectators of the game. Instead, players are always much more clearly aware of the fact that they are in a fictional situation, a game. The director of the bbfc, david cooke, sums up that the bbfc has so far considered the violence in films and games as comparable and therefore assessed them similarly. The findings of the current study, however, showed a greater distance of the players from the violence in the game than is the case with viewers of a film. Therefore, the previous classification criteria should now be revised.
The transparency of their classification criteria, praised by the kfn authors, turns out to be merely standardization in playdescription. The actual act of decision and its arguments, on the other hand, remain hidden: the discussion of which aspects are feared to affect children or adolescents of different ages, and why. As already described, the age rating of a game is made by the respective kfn tester (final report s. 22 and 91). What role the subsequent review by two kfn scientists can play in the decision-making process, whether and what discussions took place, remains in the dark.
In our opinion, transparency can only be meaningful: transparency of the procedure with corresponding possibilities of revision. We do not consider it a coincidence that kfn has failed on the ie of transparency and documentation of the decision-making processes.
Certainly, the written presentation of decision-making rationales in the usk is in need of improvement. Since 2006, a working group of the usk has been working to further qualify transparency and plausibility in the presentation of the decision and the criteria for it. To be clear: we buried the evaluation of the usk, because we are not at all of the opinion that some things could not be improved. But we wonder who evaluates the work of the kfn. Such a process had probably taken a lot of heiber air out of the discussion in the media, cooled down the level of those politically responsible and made the funding thyssen foundation think again. After all, it too has a reputation to lose.