Our rapid evidence delivery processes are outlined in Fig. 1 and discussed below in more detail. These included: 1) stakeholder engagement throughout all processes [10, 11]; 2) research question identification and prioritisation [10]; 3) bespoke phased rapid evidence review methodology [12]; 4) rapid primary research [13,14,15]; and 5) knowledge mobilisation [16].

Fig. 1

Wales COVID-19 Evidence Centre (WCEC) rapid evidence delivery processes

Organisational structure

The WCEC operated with a core team and six Collaborating Partner research groups (Table 1). The core team included a Director (AE) and leads for, research identification and prioritisation and public involvement and engagement (NJW), stakeholder involvement (AC), rapid evidence synthesis (RL), rapid primary research methods (DW), and knowledge mobilisation (MG), with managerial support (AW, JG). During the pandemic, we worked closely with members of Welsh Government’s Technical Advisory Cell (RJL) who had a boundary spanning role to promote communication between the evidence centre and policy-makers; the role continues as the Welsh Government undergoes post-pandemic reorganisation.

Table 1 Wales COVID-19 Evidence Centre (WCEC) collaborating partner research groups

The core team worked closely with a Public Partnership Group, consisting of eight members [11]. Established in March 2022 following open recruitment through Health and Care Research Wales, these individuals represented the views of the public with regards to COVID-19 research and were involved in all stages of our evidence synthesis work. They wrote lay summaries to accompany our evidence reports and co-authored our publications (AS). Additional public partners were sought specifically (through open recruitment via Health and Care Research Wales) based on the primary research topic focus [11].

The six Collaborating Partner research groups are independent Welsh research teams based in Universities or the NHS, each with their own areas of domain and methodology expertise on which to draw, depending on the research question (Table 1). A fortnightly methodology subgroup meeting included representation from all Collaborating Partner review teams for shared learning and iteration of processes. We also liaised with other national and international research partners (e.g. International Public Policy Observatory, National Institute of Clinical Excellence (NICE), UK Health Security Agency) to avoid duplication of effort and ensure complementary analyses.

Rapid evidence delivery processStakeholder engagement

Stakeholder engagement and collaboration was integral throughout our processes to ensure that we delivered research evidence that was timely, of the highest priority, and directly relevant to policy and practice [10]. Important COVID-19-related research questions were invited from various health and social care stakeholder groups during several rounds of the Stakeholder Research Question Prioritisation Exercise (ScoPE) process (described in Section 2 below). Key stakeholders were identified through an inclusive stakeholder mapping exercise and included the public, policy leads, health, education, and social care service delivery organisations and professionals [10, 11]. Further public engagement was sought via public facing symposium events in March 2021 and March 2022. We also conducted focus groups with communities that were disproportionately impacted by the pandemic to facilitate engagement, identify their priority questions and promote equity. This included black and ethnic minority groups, children and young people, housing association tenants, and disabled people [10, 11].

When a proposed question was adopted onto our work program, the relevant stakeholders (n = 2–3) and at least one public member were invited to join a series of online stakeholder meetings (usually three) with the lead Collaborating Partner research team to clarify the research question, identify the evidence need and the urgency, discuss early findings, contribute their expertise and knowledge of key articles / research, and become involved in dissemination of findings.

Research question identification and prioritisation

Our prioritisation process aimed to identify and select research questions that were of highest priority for COVID-19 focused health and social care policy and practice in Wales, [10, 11] in a situation where time did not allow for recognised formal prioritisation exercises such as James Lind [17]. Priority questions were invited through a bespoke, demand-driven Stakeholder Research Question Prioritisation Exercise (ScoPE) process via direct stakeholder consultation both within Welsh Government, and with external NHS, social care, professional, public, academic, industry and third sector groups [10]. The ScoPE process was formally conducted every six months, but was also reactive to accommodate emerging or urgent health, social care, or education research priorities needed to inform decision-making.

During the ScoPE exercise, stakeholder groups were invited to complete a proforma (please see supporting information 1) that ranked their ‘top research priorities’ (up to 10). Additional information requested included: relevance to the current or future COVID-19 context in Wales, importance of the evidence gap, potential benefits and for translation into practice, and urgency for the evidence. Submitted research questions were assessed against these criteria by the WCEC core team and public representatives for acceptance onto the work program. If necessary, further expert stakeholder advice was sought to clarify priorities and refine the research question. For efficiency, there was initial consideration of question overlap with work already undertaken or in progress (both within the Centre and externally), and whether evidence synthesis or primary research was needed. The work program was shared and discussed with Welsh Government representatives. Approved questions were then allocated to either the Evidence Synthesis Work Program or the Primary Research Work Program (see Sections 3 and 4 below).

Evidence synthesis work program

Research questions accepted onto the work program for evidence synthesis were allocated to one of our partner groups, with questions matched to experience within the group where possible. Our phased rapid review approach, [12] based on three types of products, was developed in line with international rapid review approaches to ensure we conducted and delivered robust, timely and efficient and effective evidence syntheses, [18,19,20,21,22] which also benefited from experience within the partner research groups [23,24,25].

Phase I: Rapid evidence summary (~ 1 week)

An initial introductory stakeholder meeting was set up, which included members from the WCEC core team and public representatives, the partner research group and key stakeholders. The meeting was held online and lasted about an hour. The aim was to clarify with the stakeholders the focus of the research question, how the evidence would be used, and proposed timelines.

The review team then conducted an exploratory review of key COVID-19 resources for existing reviews that may address the research question. A list of key resources was developed with information scientists to support the searches (see Supporting Information 2). This initial phase allowed the reviewers to familiarise themselves with the topic area, check the research question had not been addressed by another group and identify the likely extent and type of available evidence to inform the methods and design of the rapid review.

The output from this phase, based on abstracts and generally completed within a week, was presented as an annotated bibliography with key findings, called a Rapid Evidence Summary (RES), and discussed in a second online stakeholder meeting. This also provided the opportunity to present limited interim findings to stakeholders. If a relevant and current systematic review was identified that addressed the research question, then it could be summarised and appraised as a final product. For urgent decisions or where there was insufficient evidence to progress to a rapid review, the RES was published as the final product.

Phase II: Rapid review (1–2 months)

If sufficient evidence was identified in the RES, discussions during the second stakeholder meeting moved onto planning the Rapid Review (RR). This involved refining the research question and drafting the eligibility criteria (based on an evidence synthesis framework such as ‘PICO’) [26]. These discussions were also used to establish if there were particular equality considerations, and the potential economic impact of the evidence.

The rapid review was conducted using a variation of the systematic review approach, where components of the review process were abbreviated or omitted to generate the evidence to inform stakeholders within a short time frame whilst maintaining attention to bias. This offered the most rigorous and comprehensive product produced in a timely manner. As far as possible, the reviews followed methodological recommendations and minimum standards for conducting rapid reviews [18,19,20,21,22]. If timelines were tight, methodological decisions needed to be pragmatic. Approaches were described for transparency and included: a tertiary review (review of reviews), prioritising identified reviews for synthesis, and limiting searches for primary studies to countries with similar health and social care systems to the UK. When a focused review question could not be selected, an interim Rapid Evidence Map (REM) was conducted to support the selection of a substantive focus for the rapid review. The REM used abbreviated systematic mapping or scoping review methodology [27, 28]. The output from this phase was a rapid review report (template in Supporting Information 2) which was presented and discussed in a third online stakeholder meeting.

Primary research work program

This additional work program was set up in March 2022. Fig. 2 (an elaboration of process 4 in Fig. 1) outlines the topic identification, assessment, review and allocation processes, again designed to promote efficiency and effectiveness, and described below. Primary research projects were identified through three main routes: key gaps identified by WCEC evidence synthesis outputs; the ScoPE process (see Section 2) [10]; and though applications submitted by research groups. All questions identified via these three routes were subject to assessment against the ScoPE process described above and following criteria:

a)

Addressed the pandemic challenges (including recovery) in the context of Wales AND

b)

Built on research already undertaken in Wales and with further unanswered questions OR

c)

Utilised particular Welsh expertise for innovative work on COVID-19 illness, impacts & recovery OR

d)

Was a high priority question, with clear policy implications for the context of Wales.

Fig. 2

Wales COVID-19 Evidence Centre (WCEC) Primary Research Work Program topic identification, assessment, review, and allocation processes

All primary research needed to be deliverable in about 6 months, not be more suitable for alternative funding streams, and offer value for money and potential for impact. If criteria were met, researchers were invited to complete a full application form, which included further details regarding methods, pathway to impact and costings. Once received, the full application underwent peer review by one internal reviewer, one external reviewer (with topic / methodological expertise) and two public members. A grant funding panel met to discuss the applications, and successful projects received funding for the work.

We also had core team capacity and expertise to support a range of ‘in-house’ rapid primary research projects. Where the in-house team did not have the relevant specialist expertise, we commissioned appropriate Collaborating or external partners to conduct the work. For in-house and commissioned research, a similar process to the evidence synthesis program was used: three online meetings with the key stakeholders to clarify the research question, identify appropriate methods and analysis, provide expertise in the research process, and assist with knowledge mobilisation and the pathway to impact.

Knowledge mobilisation and impact

Our knowledge mobilisation processes were designed to ensure our products were accessible, timely and useful for our stakeholders to inform policy, practice and decision-making to promote effectivenss [16]. The process was iterative and tailored to meet the requirements of the stakeholders. The third online stakeholder meeting was used to present the findings from the evidence synthesis or primary research, address any queries, and support the development of a knowledge mobilisation plan, co-designed with the proposing stakeholders.

The templates for our rapid review final reports (Supporting Information 2) were based on recommendations for reporting evidence reviews for policy-makers and have been adapted for our primary research reporting [29]. For each report, a 'topline summary’ was developed highlighting the methodology, evidence base, research quality, key findings and implications for policy and practice. The report’s findings were also re-drafted into a lay summary by our public representatives to provide a widely accessible version, published alongside other outputs including infographics. Reports were published on pre-print servers and linked to the lay summaries on our WCEC website library.

Activities to promote the uptake and use of the evidence included fortnightly internal Welsh Government evidence briefings, where research findings were presented to a wider Welsh Government audience and invited key stakeholders. Here, implications of the evidence and practical next steps towards implementation were also discussed. A communication plan was in place, and we used social media through Health and Care Research Wales to disseminate findings including infographics and links to our review outputs and newsletters. We also recorded and tracked the impact of our work via ongoing engagement with stakeholders and an online survey. Our reports were made publicly available on our website, which was also linked to other COVID evidence resources (‘UK Health Security Agency- COVID Rapid Review Collections’ and internationally with COVID-END) [20, 30].