moderation and quality criteria
The project and (training) resource profiles that you can find on the platform have been submitted by the citizen science community or the EU-Citizen.Science project consortium. Before publishing, they undergo a moderation process where the quality criteria framework developed within the EU-Citizen.Science project is applied. Below you can find a description of the moderation process and of all the criteria. IIASA and other partners in the project have developed these criteria and assessment framework as part of WP3 “Content - Framework, Quality Assurance and Curation” of the EU-Citizen.Science project. The work carried out can be found in D3.1 Framework Report Describing Criteria and Rationale for Sharing and Selecting State of the art Citizen Science Resources and D3.3 Review of Framework Implementation Defined in D3.1.
The moderation process
Our ambitious goal is to become the place to share useful resources about planning, running, evaluating and participating in citizen science initiatives or simply learning about citizen science, including tools and guidelines, best practices and training modules. We hope to make practical guidance on citizen science findable and accessible to all, and enable people to initiate their own activities wherever they are. It is important to us, and the community of practitioners, that we have a way of ensuring that the resources profiled on the platform are of good quality, and that the projects profiled on the platform are indeed citizen science initiatives. This is why we have a moderation process for all submissions.
When you create a profile for a resource or a project to share on the platform, you will be asked to provide a range of descriptive information about it, and once all of the mandatory data fields are completed, you will be able to submit the profile. Upon submission, it will be sent for moderation. Until it has been reviewed against our criteria, it will be marked as ‘not yet moderated’, and will not be automatically visible in search results. Once your resource profile has successfully been through the moderation process, it will be marked as ‘moderated’, and will be fully available in search results.
Quality criteria
We have developed a set of comprehensive quality criteria as a way of ensuring that the projects and resources that you can find on this platform are of good quality. The overarching criteria can be applied to both projects and resources. Specific and supporting criteria have been developed for resources. The table below provides an overview of all resources; further down on this page you can read about all the quality criteria in more detail.
| Overarching Criteria (for projects and resources) | About citizen science or relevant to citizen science (required) | |
|---|---|---|
| All mandatory metadata is provided (required) | ||
| Engages with the ECSA 10 Principles of Citizen Science (suggested) | ||
| Specific Criteria (for resources) | Access to the resource | |
| Readability and Legibility | Clear structure | |
| Clear language | ||
| Basic formatting | ||
| Content | ||
| Applicability | Easy to implement | |
| Easy to adapt | ||
| Object | Clearly audible (audio object) | |
| Good quality (video or image) | ||
| Supporting Criteria (for resources) | Evaluation | Used in the context of a citizen science initiative |
| Evaluated in terms of the quality of the content, methods, or results of the method | ||
| Impact | Impact it has had/could have | |
| Measured impact | ||
The overarching criteria for all resources
Required Criteria - Citizen Science
The first criterion that the moderator will look for, is that your resource is relevant to citizen science, or that your project is a citizen science initiative.
Although there are no hard and fast definitions of what citizen science is and is not (nor should there be - citizen science should always remain a broad and inclusive concept), we turn to the recent work conducted by the citizen science community on the ECSA Characteristics of Citizen Science as guidance. Please consult these characteristics if you are uncertain whether the resource or the project you would like to share really does relate to citizen science. If the moderator has concerns they will contact you directly.
The moderator will also check if all mandatory fields in the resource profile are completed. This is to ensure that all project and resource profiles provide sufficient information to enable users of the platform to see whether it is useful or relevant to them.
Suggested Criteria - the Principles of Citizen Science
The moderator of your submitted profile will also consider whether your resource or project is in alignment with the ECSA 10 Principles of Citizen Science.
Specific criteria
For the resources to be shared on the platform, we have developed a specific set of further criteria, which will be assessed during the moderation process. You can read more about how we developed these in the blog post ‘How we developed the quality criteria for resources’.
We consider good quality citizen science resources to be those that are easy to access, implement and adapt, well structured, clearly described and written in clear language, and ideally improve or support the desired impact of the initiative (eg. on science, policy or society, etc). For this purpose we have developed a range of criteria that the moderator will assess against a 5-point scale, from strongly disagree (1 point) to strongly agree (5 points).
- Easy access to the resource
- The resource should be easy to access, i.e. it doesn’t require registration, and is not behind a paywall.
- Readability and Legibility
- The resource should be clearly structured according to the type of the resource. For example, a scientific paper or report should include an introduction, methodology, results, discussion and/or conclusions, and methodology documents should include an introduction, audience description, step by step methodology, and an example.
- The resource should be written in clear language that is easy to read and understand for the intended target audience, and should be concise, unambiguous, and avoid the use of unusual words and jargon. Where technical terms are used, their meaning should be explained clearly.
- The resource should pay attention to basic formatting, such as clear titles and paragraphs, correct grammar and spelling, a legible font of large enough size to read, and clearly marked references.
- Clarity of Content
- The resource should clearly describe its aims, goals and methods, so that it is easy for readers to understand how to apply the resource in their own context.
- Applicability
- The resource should be easy to implement, ideally with descriptions of how it can be implemented, the contexts that it is useful for, and recommendations for further use or development.
- The resource should be easy to adapt to different cases, ideally with an explanation of any limitations of the resource and the contexto in which it could be useful, and with guidelines or recommendations for its adaptation to different cases.
- Object Quality
- If the resource is an audio object, it should be clearly audible, with no interruptions or background noise.
- If the resource is a video, an image or illustration, the quality should be good enough to see clearly, with a sharp focus.
The highest possible score is 40 points, if all criteria are applicable. But in most cases, only a selection of the above criteria will be applicable. The moderator will look to see that your resource exceeds 50% of the total possible points for that type of resource. That is the threshold for being listed on the platform as a good quality resource.
Supporting criteria
It really enhances the value and quality of your resources when you can say something about how it has been used and developed further in practice, and whether or not it has been evaluated for usefulness and applicability in practice. The moderator will also take it into positive consideration if there is evidence of the following criteria being met (but these are not part of the threshold calculation mentioned above):
- Evaluation
- The resource has been used in the context of citizen science, or is currently being used in a citizen science initiative, and the outcome of this has been shared.
- The resource has been evaluated in terms of the quality of the content, or the methods, or the results of the method, and the outcomes of these evaluations have been shared.
- Impact
- The resource refers to any impact that it could have (or has had) on science, policy, society, etc.
- The impact of the resource has been measured and is shared in the resource.