Multi-criteria decision analysis to evaluate fluoride-removal options in Ethiopia

NOTE: Article from the Geogenic Contamination Handbook

Background

Multi-Criteria Decision Analysis (MCDA) is a technique for comparing and evaluating different options (or measures) in order to identify options with the broadest acceptance, or to rank options or to distinguish acceptable from unacceptable options. In a workshop, stakeholders with different perspectives (e.g. regional government agencies and householders) select criteria important to them which they can use to compare different options. A list of criteria is then made that all stakeholders can accept, and then the options are valuated with the help of the criteria. The list is interactive and facilitates transparent and participatory assessment. MCDA can foster collaboration and learning in a situation in which a diversity of interests are openly represented. There are different approaches within the MCDA family. The selection of commonly used approaches, which include Multi-Attribute Value Theory (MAVT), Multi-Attribute Utility Theory (MAUT), Analytical Hierarchy Process (AHP) and the Simple Multi-Attribute Rating Technique (SMART), depends on both the nature of the question and the experience and educational level of the stakeholders involved (Kiker et al., 2005). The MAVT is one of the most commonly used approaches, partly because it has conceptually straightforward procedures that are relatively easily understood (Karjalainen et al., 2013). The MAVT procedure consists of following steps:

1. Establishing the decision context
2. Identifying the options
3. Identifying objectives and criteria
4. Scoring
5. Weighting
6. Obtaining an overall value
7. Calculating values
8. Examining the results

Here we illustrate the MAVT procedure used in a workshop to evaluate different fluoride removal technologies in Ethiopia. See Osterwalder et al. (2014) for a description of MAVT procedure and technical information presented at the workshop. The purpose of the workshop was to bring the different stakeholders together to discuss fluoride-removal options for drinking water and what factors, particularly cost, need to be considered when selecting a method for implementation.

Procedure

Step 1. Establishing the decision context

Table 9.1 Parameters of the three scenarios (Terms of use: Cite original source from Handbook)

Information is needed by decision-makers as a basis for deciding among alternatives. The decision context determines to some degree what information is required. The decision context is governed by policy, administrative and technical issues and the social context. Stakeholders and other key players who should be involved in the decision context need to be identified, as does the extent of their participation in the analysis. Not all stakeholders need to participate physically in the MCDA, but their values should be represented by one or more key players who do participate. The decision context is decided on at the beginning; e.g. “Sustainable fluoride-free water solutions for rural households in Ethiopia”. On 27th April 2012, a one-day stakeholder MAVT workshop was held in Addis Ababa with around 40 representatives from the federal government, regional governments, non-governmental organisations and academia. The aim was to assess fluoride-removal technologies appropriate for rural Ethiopia. Workshop participants were asked to evaluate the technologies for each of three scenarios (Table 9.1) with different fluoride concentrations, water consumption and water scarcity. In addition, the acceptance of bone char filter material and water salinity were considered. In plenary discussion, stakeholder groups evaluated the different options for the three scenarios using the MAVT approach. Because of time constraints, the research team preselected technologies and criteria based on interviews with 10 institutional stakeholders held early in 2012. Background information for each technology, for example, costs and technological requirements, were also collated in preparation for the workshop.

Step 2. Identifying the options

Fig. 9.9 Selected fluoride removal technologies (Terms of use: Cite original source from Handbook)

The options within the decision context need to be selected. The five technologies illustrated in Figure 9.9 were selected for the MAVT exercise. Provision of fluoride-free water was excluded because, although it is the long-term option of choice for the National Fluorosis Mitigation Project Office (NFMPO) of the Ministry of Water and Energy in Ethiopia, in the short term, it is fluoride-removal options that are needed. Technologies that have been implemented in other countries (e.g. electrocoagulation; Gwala et al., 2010) or which are under development in Ethiopia (e.g. aluminium oxide; Shimelis et al., 2005) were not considered here, as the stakeholders present at the 2012 workshop thought it would be premature to include them. An important criterion for the selection of

Step 3. Identifying the objectives and criteria

Table 9.2 Criteria selected in Step 3 for the assessment of fluoride-removal technologies in Ethiopia

To be able to rate and compare the different safe water options, a number of criteria need to be agreed on. These could, for example, be the costs involved, their technical performance, their accessibility to all in the community or the lifespan of technologies or machinery involved. Three objectives with which to compare the different options – reliability, acceptability and affordability – were identified together with measurable criteria (Table 9.2).

Step 4. Scoring

Table 9.3 Criteria attributes and scores for the reliability objective for Case 1 (Step 4). In Case 1, RO is excluded because of water scarcity.

The next step is to determine values for the criteria and to give them comparative scores. Each evaluation needs to be turned into a score. Normally, the scale extends from 0 to 1, 10 or 100. This is necessary in order to be able to combine different types of values, for example numerical values and qualitative ratings (poor, medium, good). More detail on scoring can be found in the MCDA manual of the Department of Communities and Local Government (2009). As an example, criteria attributes and scores are shown for Case 1 (Table 9.3). In the case study presented here, the stakeholders strongly objected to the 0 value for the minimum score and weight. We therefore assigned the scale from 1 to 10 for both score and weight. The sensitivity analysis indicated that this scale range did not alter the final ranking of different options. The background to criteria attributes and scores is given in Osterwalder et al. (2014). It should be noted that some attributes are location-specific, while others are not. Exclusion factors also needed to be considered, as not all technologies are suitable for all settings. These were:

• if the total cost of producing treated water were to exceed 100 ETB/m3
• if less than 70% of the target population were to accept the technology, either for cultural or religious reasons or because of taste
• if fluoride concentrations < 1.5 mg/L could not be achieved
• if major interventions were to be necessary less than every 60 days
• if the technology were to produce a high volume of contaminated, non-potable water in a water-scarce area. This applied primarily to RO.

Step 5 Weighting the criteria

Table 9.4 Weighting of criteria by the different stakeholders (most important = 10)

Stakeholders assign weights to each of the criteria to reflect their relative importance for the decision. Usually, different stakeholder groups will have different opinions on the importance of the various criteria and will therefore assign weights differently. In this example, we are using weights on a scale from 1 to 10. The most important criteria will therefore be assigned a weight of 10 and the least important a weight of 1, with the remaining criteria weighted in between. The results of criteria weighting revealed that the local availability of raw materials and simplicity of operation are major points of concern. Further, a majority of the stakeholders put a high priority on the fact that the Ethiopian national guideline needs to be met, and the treated water needs to be accepted by the consumer. Different stakeholder groups prioritised different criteria in different ways (Table 9.4). Participants from central government authorities considered capital costs more important, while the representatives of local governments put a higher priority on operational costs, reflecting the fact that the central government often pays for construction, leaving local governments to supervise operation and management. Academics and, to a lesser extent, NGOs and development partners, tended to place a higher priority on aesthetics and a lower priority on costs, perhaps reflecting concerns about sustained use.

Step 6. Obtaining an overall value

The scores for each criterion are multiplied by the given weights to gain one final, overall value for each mitigation option. This can be mathematically expressed as V(A)=∑w_i·v_i (a_i) In the above equation, the scores given for each criterion (v_i(a_i)) are multiplied by their given weights (w_i), and these weighted scores are then summed up to gain the final, overall value V(A) for mitigation option A.

Step 7. Examining the results

Table 9.5 Ranking of preferred options using the average weighting for Case 1

Fig. 9.10 Stakeholder preferences for Case 1 (Terms of use: Cite original source from Handbook)

The results can be examined to determine the ranking of options. The results of the MAVT study are given in Table 9.5 and Figure 9.10. In Case 1, high costs and water scarcity resulted in the exclusion of RO. In Case 2, the high fluoride content was the cause of the elimination of BC, as the filter material would have to be replaced too often. In cases 2 and 3, NT was excluded because the WHO guideline value of 1.5 mg/L could not be achieved. Because of the relatively high water requirements and elevated fluoride concentrations, filtration was not suitable, leaving only RO as the remaining option.

Step 8. Sensitivity analysis

The calculated results of an MCDA may be sensitive to changes in the scores and weights assigned to the options and criteria. Even small changes in weighting or scoring may lead to a completely different option being the “preferred option”. In projects that attract public interest, the choice of weights may also be controversial. A sensitivity analysis can highlight these kinds of problems and provide a means for examining the extent to which vagueness about the inputs, or disagreements between stakeholders, makes a difference to the final results. The MCDA manual of the Department of Communities and Local Government (2009) describes details on how to undertake a sensitivity analysis.

Conclusions and feedback

Although there was a little scepticism at the beginning of the workshop, all agreed at the end of the day that the workshop had been very useful. First and foremost, the participants were of the opinion that it had been useful to have quantitative data that allowed them to discuss and compare different options objectively. Secondly, the participants valued being able to see for themselves that there is no single, most preferable technical solution for fluoride removal in Ethiopia and that the selection of a technology depends on location-specific parameters and on the relative importance put on different criteria by the stakeholders involved. Thirdly, it was interesting for all to note that there was good agreement between stakeholders in the selection of options. The necessity of examining different financing strategies also became clearer through the separation of costs (into capital, capital maintenance and operational costs). In the absence of an MAVT, different sets of stakeholders tend to prioritise one option, perhaps because their organisation is promoting it. Other stakeholders may exclude one option by considering only one single criterion, sometimes without the support of empirical evidence, e.g. “reverse osmosis is too expensive” or “bone char is not acceptable to consumers”. The MAVT exercise helped to provide a more objective view of the different options. Stakeholder groups could argue for different weightings for different criteria, but not for specific technologies. Ideally, the MAVT procedure should be repeated with all stakeholders, as more information on existing technologies, or on new ones, including fluoride-avoidance options, becomes available. The methodology can easily be expanded to include more information about conditions specific to a particular location.

References

For references, please visit the page References - Geogenic Contamination Handbook.