Summary

In a risk assessment process as defined in international standard ISO 12100:2010, risk estimation is an essential step in which machinery designers and users determine risk levels and identify the most critical hazardous situations. Two previous studies funded by the Institut de recherche Robert-Sauvé en santé et en sécurité du travail (IRSST) revealed that the many risk estimation tools available take a wide variety of forms and that a number of their characteristics (e.g., parameters, architecture) can have a significant influence on the estimated level of risk. In this third instalment of the IRSST’s research program on the assessment of machine risks, the impact of these characteristics was evaluated and a number of risk estimation tool design principles were validated by means of an experimental study involving a variety of users, chiefly from industry.

Six tools were analysed: 4 that were selected from a list of 31 drawn from earlier studies, as well as tools provided by the IRSST, the Commission des normes, de l’équité, de la santé et de la sécurité du travail (CNESST), and the Health and Safety Laboratory (HSL) in the United Kingdom. Some risk estimation parameters having flaws or potential biases were also selected. The risk estimation tools and parameters were applied to concrete scenarios of machinery-related hazardous situations by a representative sample of industry stakeholders, occupational health and safety officers, joint sector-based association advisors and trainers from various sectors. They were all asked to give their impressions and preferences with regard to the different risk estimation tools, along with their reasons.

With respect to the study of risk estimation parameters, five of the potential flaws established in the preceding instalment of this research program were analysed. The experimental results show that when subjects have trouble applying a parameter, they are generally able to associate it with a flaw in the parameter. The results also indicate fairly clearly that the impact of parameter design flaws is not uniform. The type of flaw, its position on the parameter scale and the scenario in question influence its impact on the determination of the level of a parameter. The severity of harm parameters seem fairly robust and allow users to reach a solid consensus, despite the presence of flaws. On the other hand, the parameters for the likelihood of harm occurring and the likelihood of the hazardous event occurring are decidedly less robust. Besides the flaws in these parameters, the results obtained suggest that evaluating likelihood is a difficult aspect of risk estimation that requires special attention.

For the study of the risk estimation tools, three criteria were used to determine potential application problems: (1) the ability to distinguish between scenarios with different risk levels, (2) the user’s satisfaction with the result obtained and (3) the convergence of results (intersubject repeatability). By determining the origin of the problems found with the six risk estimation tools, the effect of the failure to follow certain design principles for tool architecture could be confirmed. An architecture that gives more influence to a specific parameter (e.g., the first parameter on a chart) may amplify a divergence in results and reduce the ability to classify scenarios appropriately, especially if there are flaws in the most influential parameters. A matrix sensitive to the slightest change in the level of a parameter will have the same impact when there is a flaw on one of its parameters.

An architecture that did not produce a uniform distribution of risk levels led to unsatisfactory results and to problems of scenario discrimination. A structure that did not comply with ISO 12100:2010 did not reveal any impact on the risk estimation process.

It is hoped that these findings will help improve the robustness and reliability of existing tools and provide support for the risk assessment training currently provided by partners.