Skip to main content

Table 5 Differing interpretation about level of intervention being assessed and related implementation failure

From: Applying the ROBINS-I tool to natural experiments: an example from public health

Provided below are quotes from two included studies on study aim(s) and interventions

Braubach [12]*

Study aim: “to assess potential health impacts of improved thermal insulation. Key objectives were to assess the impact of thermal insulation changes on indoor environments, and evaluate potential effects on residents’ health.”

Intervention description (may include possible co-interventions): “thermal insulation of all building facades; thermal insulation of the roof/ceiling of highest dwelling; thermal insulation of basement/floor of lowest dwelling; installation of energy-efficient windows where replacements were necessary; installation of new heating systems in buildings with substandard systems. Additional renovation projects without significance for thermal comfort were painting of staircases, installation of intercom systems, new power and water supply systems, improvement of outside spaces/greenery and other repairs as required. However, these renovations are not part of the survey and their impact will not be looked at, although they may improve the general quality of the dwelling significantly.”

Shortt [13]*

Study aim: “The evaluation focussed on two elements of the process firstly, assessing the benefits to households in terms of indoor environment, health and wellbeing and household income…. This paper focuses primarily on the installation of central heating in selected households and the immediate effect on the dwellings and their occupants”

Intervention description (may include possible co-interventions): “installation of central heating systems and other energy efficiency measures in homes…The overall aim was to develop an energy efficiency programme in partnership with key agencies and local communities and as a result to increase energy awareness, increase uptake in grants and reduce the adverse effects on health and well-being caused by cold homes.” The intervention also involved “encouraging higher uptake of social security benefits.”

*In both the above examples, the intervention was tailored according to the need of the individual household, but details of this was not reported and was not controlled for in the analysis. In addition, subject to participants’ own resources, the interventions were available to participants regardless of participation in the study, raising the potential for contamination. Again, this was not reported on.

Possible levels of implementation: potential for implementation failure/variation in adherence**

Programme level: selective uptake by eligible households or by external factors e.g. changes in funds available to those delivering intervention

Operational level: incomplete delivery or installation of intended intervention(s). Successful implementation may also require an educational component to ensure recipients know how to use intervention effectively and have aspects such as potential benefits and costs explained.

Household level: householders in receipt of intervention do not use intervention as intended—heat more rooms but with same cost. Impacts are dependent on behavioural change.

**External factors can also affect implementation and/or adherence such as changes in fuel costs.