Validation of the Measuring Instrument by Experts
To validate the content through expert judgment, a large group of experts from different countries was selected. They were asked to assess the appropriateness of each item for measuring social exclusion, according to the operationalisation carried out of such concept, as well as for the clarity of each item.
A) Selection of experts
In this phase experts in criminal law and/or criminology were located and selected from 18 different countries, mostly North America and Europe. Specifically, they came from: Belgium, Canada, Denmark, Finland, France, Germany, The Netherlands, Hungary, Ireland, Italy, New Zealand, Poland, Portugal, Rumania, Spain, Switzerland, the United States and the United Kingdom.
This decision was based on the project’s objective of designing a measurement instrument for application in the Western developed world, and it encompasses the following geographical areas: Central Europe, Eastern Europe, Northern Europe, Southern Europe, and English-speaking countries in Europe and outside Europe.
B) Validation by international experts
- 1st phase of validation (experts with at least 8 years of experience). Each expert was sent one third of the items from the instrument to be validated, selected randomly, and they were asked to reply to a dual Likert scale about the appropriateness and clarity of each item. The statistical analysis of the responses allowed us to eliminate the items that had not been validted due to the lack of consensus regarding the item’s appropriateness and clarity. This first validation gave rise to a new tool, smaller than before, that then underwent a second phase of validation.
- 2nd phase of validation (experts with at least 20 years of experience). In this phase the number of items had been considerably reduced, so the experts were sent the reduced instrument, either whole or divided into two parts, for validation and for assessing the appropriateness of the item to the pre-established dimensions (according to Aiken’s V coefficient). After eliminating the items not validated by these experts, the final instrument was obtained.
C) Weighting of the indicators of the final instrument
Given that not all items have the same weight or importance in relation to the construct to be measured, they had to be given relative weights in relative terms (weighing), once the final instrument was obtained. For this purpose the members of the research and work teams at this point in the investigation assigned scores to each item by completing a questionnaire. The result of this analysis was submitted to the consideration of the research and work teams, who reached a consensus regarding the weight of each item in relation to the others. This was how the items comprising the instrument were internally weighted.