DEVELOPING CONSEQUENCE THRESHOLDS FOR STORM MODELS: CASE STUDY OF WESTERLY RHODE ISLAND

Emergency managers face challenges in understanding and communicating potential hurricane hazards. Preparedness typically emphasizes the last event encountered, the potential implications of future hazards may thus be underestimated. Risk assessment models (e.g., basic HAZUS) that emphasize accumulated damages in economic terms do not provide actionable data regarding specific local concerns, such as access by emergency vehicles and potential communications disruptions. Qualitative methods conventionally used to identify these concerns, however, lack the specificity necessary to incorporate the managers’ knowledge into hazard models (e.g., highly exact geographic location of the vulnerability or cascading consequences). This research develops a method to gather rich, actionable, qualitative data from critical facility managers that can be utilized in combination with hydrodynamic, wind, and precipitation models to assess potential hazard impacts. A pilot study was conducted with critical facility managers in Westerly, Rhode Island USA, using semi-structured interviews and participatory mapping. Interview methods were based on existing practices for vulnerability assessments, and further augmented to obtain data based on hurricane modeling requirements. This research identifies challenges and recommendations when gathering critical facility manager's knowledge for incorporation into storm simulations and contributes to the overall participation and knowledge co-generation framework of natural hazard models.

economic terms do not provide actionable data regarding specific local concerns, such as access by emergency vehicles and potential communications disruptions.
Qualitative methods conventionally used to identify these concerns, however, lack the specificity necessary to incorporate the managers' knowledge into hazard models (e.g., highly exact geographic location of the vulnerability or cascading consequences). This research develops a method to gather rich, actionable, qualitative data from critical facility managers that can be utilized in combination with hydrodynamic, wind, and precipitation models to assess potential hazard impacts. A pilot study was conducted with critical facility managers in Westerly, Rhode Island USA, using semi-structured interviews and participatory mapping. Interview methods were based on existing practices for vulnerability assessments, and further augmented to obtain data based on hurricane modeling requirements. This research identifies challenges and recommendations when gathering critical facility manager's knowledge for incorporation into storm simulations and contributes to the overall participation and knowledge co-generation framework of natural hazard models.
iii ACKNOWLEDGMENTS I want to thank my advisor, Austin Becker, for guiding me through a series of independent research projects that defined my graduate school experience. The freedom he gave me and the international coastal resilience projects he exposed me to allowed for the inspiration of this paper and fueled my passion to prepare communities for climate change. Many thanks to Peter Stempel for developing the methods that enabled this research and for pushing me to become a better thinker.    3 Data required to make qualitative concern data actionable in DRR models.   (Romero & Emanuel, 2017). To address these risks coastal managers employ Disaster Risk Reduction (DRR), the systematic practice of evaluating and reducing risks posed by natural hazards such as storm surge associated with hurricanes (Thomalla, Downing, Spanger-Siegfried, Han, & Rockström, 2006). Incorporating stakeholders concerns through processes such as workshops can improve the accuracy and usefulness of DRR assessments (Messner & Meyer, 2006), and is an essential component of hazard management (Eakin & Luers, 2006).
Current DRR assessments typically use single average vulnerability curves representing damage as a function of depth of flooding, an approach that does not account for more detailed qualitative concerns that may be raised by stakeholders (Aerts et al., 2018). Such concerns include, for instance, the loss of a diary or the short-circuiting of a hospital's generator. In many cases, DRR assessments that account for social factors are not at a fine enough geographic scale for practitioners to use in DRR projects (Carr, Abrahams, Arielle, Suarez, & Koelle, 2015).

Storm impact model definition:
A form of hazard impact model that allows a storm simulation to interact with stakeholders concerns (Figure 1). Storm impact models that combine an individual's qualitative concerns with storm simulations (e.g., the ADvanced CIRCulation model (ADCIRC)) at the intra facility scale have been developed (Stempel, Ginis, Ullman, Becker, & Witkop, 2018).
Storm impact models can incorporate a wastewater treatment plant operator's concern that an aeration tank will spill during a storm surge and spread disease within the community it serves. However, methods for gathering qualitative concerns for incorporation into storm impact models at the intra-facility have not been elaborated (Hendricks et al., 2018;Messner & Meyer, 2006;Aerts 2018;Palmer, 2014).
Gathering qualitative concerns for incorporation into storm impact models is complex.
It requires that aspects of the concern, such as the wind speed at which the concern occurs, be defined in quantifiable terms. For example, incorporating a fire chief's concern that a generator will short circuit due to flooding into a storm impact model requires the generator's location along with the water level that would cause the generator to short circuit.
This research thus seeks to establish and test a method to gather rich qualitative concerns (Doody & Noonan, 2013)  The basis for the method tested is drawn from and expands current approaches to vulnerability assessment that address aspects of disaster risk and public engagement related to DRR. After establishing the basis for this approach in the background section, the methodology is subsequently evaluated based on interviews with 15 facility managers (FMs) in Westerly Rhode Island. The paper identifies challenges that exist when gathering FM's concerns for incorporation into storm impact models and how storm impact models can be improved by including stakeholders in storm impact models through participatory assessments.

Need for a method to develop actionable qualitative data for models used in disaster risk reduction.
An increasing body of literature explains that planners should understand and include the concerns of stakeholders (defined here as potentially vulnerable individuals) in the assessment and planning processes when preparing for hazards (Becker, 2013;Paul et al., 2018). This is in part because concerns influence how people behave (Moser & Ekstrom, 2010) and the behaviors of individuals, businesses, and government entities before, during and immediately after disasters dramatically affect impact and recovery (Aerts et al., 2018). To illustrate, when behaviors are ignored during DRR assessments, flood risk can be overestimated by 100% (Haer, Botzen, Moel, & Aerts, 2017). A behavior includes specific responses that reduce exposure (e.g. moving out of harm's way or shutting down a facility). These behaviors are kinds of qualitative data that can be gathered and incorporated into storm impact models. Specific concerns and perceptions vary based on social and cultural factors, therefore, vulnerability varies based on social and cultural factors (Nielsen & Reenberg, 2010). Accounting for concerns in hazard assessment and planning is therefore beneficial.
Although concerns can increase DRR assessment accuracy, most DRR assessments ignore stakeholders concerns because they are difficult to quantify and incorporate into models (Aerts, 2018). Concerns regarding high-order losses (Rose, 2004) or hidden costs (Hienz, 2000) like damages to ecosystems and cultural assets are rarely accounted for even though the loss of these intangible objects are normally what people bemoan the most after a flood (Green, Wierstra, Penning, & van der Veen, 1994). Instead, most flood risk assessments use single average vulnerability curves representing depth of flooding and damage (Aerts et al., 2018) including HAZUS, the most common DRR assessment tool in the USA (Banks, Camp, & Abkowitz, 2014). In addition to the possibility of the storm impact model output being inaccurate (Stempel et al., 2018) or irrelevant (White et al., 2010), the lack of stakeholder input has serious ramifications for the perceived legitimacy and credibility of modeled outcomes (Schroth, Pond, & Sheppard, 2011).

Stakeholder participation and the understanding of their concerns is needed in order for planners to be effective in increasing resiliency of social environmental systems to hazards
Directly involving stakeholders in model building processes increases the quality of decisions the model is meant to assist (Salter, Robinson, & Wiek, 2010). Active, iterative and inclusive Particpatory Integrated Assessment processes that build models increase the credibility, relevance and legitimacy of the model, and therefore enhances the models ability to connect scientific knowledge with environmental policy (White et al., 2010). Guiding a participant through a boundary object in a workshop or including participants' opinions in the model itself allows participants to shed the 'blackbox' perception of the model and the model's assumptions (Salter et al., 2010;Sheppard et al., 2011;White et al., 2010). In other words, incorporating local knowledge into climate models has allowed stakeholders to validate the models, without having to understand the underlying model's mathematics (Salter et al., 2010).
As previously discussed, methods exist for combining stakeholder concerns at the intra-facility scale in storm simulations to create storm impact models that address these concerns (Stempel et al., 2018), providing that the information gathered is defined sufficiently (e.g. spatial location, depth or velocity at which the concern occurs) ( Stempel et al., 2018). The challenge thus becomes gathering qualitative data with adequate definition.

Data required to make qualitative concern data actionable in DRR models
Combining stakeholder concerns with storm simulations requires several points of data in addition to the qualitative data related to the concern (Figure 2). These five required components are explained below and when combined, make up a complete "Consequence Threshold" (CT).
1. The concern -An asset or operation the FM perceives can be directly impacted by a storm force. For example, a generator that can be impacted by flooding.

The accurate location of concern -
The latitude and longitude of the asset or operation the FM believes can be directly impacted by storm forces. For example, a generator's location at 41.12345 N and -71.12345 W. (Stempel et al., 2018).
3. The modeled hazard -A storm force that can be quantified such as inundation depth or wind velocity that is modeled as part of a storm simulation (Stempel et al., 2018).
4. The increment -The storm force threshold that when exceeded at the location of concern triggers an impact. For example, winds above 100 mph or flooding above 1 foot. (Stempel et al., 2018).
5. The impact -The outcome if the storm force exceeds the increment at the location of concern. For example, the generator would short-circuit.

Limitations of publically available CTs
These requirements were initially developed as part of work supporting of a impact reports based on combinations of storm simulations and qualitative data (Stempel et al., 2018). To gather specific outcomes for the IEMC, we turned to Rhode Island's publically available vulnerability information for critical facilities. We determined when critical facilities would be damaged because including when critical facilities would be damaged into hurricane training simulations may better prepare EMs for storms (Brecht, 2007).
The E-911 database includes the latitude and longitude for RI's schools, fire departments, police stations, hospitals and other critical facilities around Rhode Island.
This database of coordinates helps responders reach these buildings during times of emergency. Since databases are developed based on the particular purpose they are supposed to serve (Liu & Palen, 2010) it is unlikely that these points are at the location within the facility where an impact could occur. Therefore, using these points as vulnerabilities may mislead the storm impact model output because ground elevations are extrapolated based on the latitude and longitude of the given point which are in turn used to see if a critical facility is flooded (Stempel et al., 2018). Therefore, a critical facility may be marked as safe by a storm impact model, but really should have been marked as damaged by flooding because the location of concern is at a lower elevation than the given point ( Figure 2). Since this intra-facility level vulnerability information was not publically available, we decided to collect it ourselves.

Figure 2. Current publically available location and fictitious CTs for a WWTP in
Westerly, RI. Note the publically available location is not impacted by modeled flooding while two CTs have already been marked as damaged.

Methodologies we adapted to gather CTs
Methods for gathering CT components for use in DRR models such as storm impact models are not well elaborated. For instance, there is a framework to incorporate citizen scientists into hydrological models, including flood management Buytaert et al. (2014), these models however do not address the diversity of hazards encountered in storms. Other researchers give guidelines for creating realistic climate change visualizations based on citizen and expert input for use in planning processes (Sheppard et al., 2011). This includes guidelines for engaging community leaders to ensure the outputs are of locations, themes, viewpoints, and local conditions the community cares about. Although this guidance is clear and generalizable, it does not provide methods to elicit the specific forms of data necessary for storm impact modeling including the storm force (increment) that would cause a facility damage.
This research adopts principals from the vulnerability assessment (R J Cox, 2013) and participatory mapping methodologies (Cadag & Gaillard, 2012) to create a method that gathers actionable concerns for storm impact models. Some vulnerability assessments address all necessary aspects for incorporation into storm impact models, but none has reflected the gathered information in a storm impact model (Table 1).
Since no single study has explored all required CT components for incorporation into a storm impact model this research pilots the methodology presented here. Our method draws on principles from the methodologies identified in Table 1, including using open ended questions to determine stakeholder concerns, storm force increments and associated impacts (Willows, Reynard, Meadowcroft, & Connell, 2003) along with participatory mapping techniques (Cadag & Gaillard, 2012).

Case Study Westerly Rhode Island Site Selection
The early stage of gathering FM's concerns for incorporation into storm impact models to begin to answer nuanced questions made a case study an appropriate method for exploratory work (Becker, 2013). We partnered with RIEMA to give the study credibility and gain access to FMs within the case study that might otherwise be hesitant to participate because of security threat concerns (Rinaldi, 2004). We selected critical facilities in Westerly (RI) to pilot this approach because RIEMA wanted to analyze Westerly's FM's storm concerns and determine the cascading effects of a storm on those concerns. Since modeling can provide insights into critical facilities interdependencies and recovery after extreme events (Rinaldi, 2004) and Westerly has a relatively small number of critical facilities, RIEMA deemed Westerly .

Sampling approach
Gaining access to infrastructure vulnerability information is challenging because it is normally proprietary in nature (Rinaldi, 2004). Therefore, RIEMA introduced us to Westerly's local emergency managers (EMs) who, in turn, provided contact information for FMs. By the nature of their work, local EMs are highly informed about how a storm affects their community (Newkirk, 2001) and are well connected to the FMs in their community. We used FEMA's definition of a RI critical facility as a facility that, "if severely damaged, would reduce the availability of essential community services necessary to cope with an emergency" and "a facility associated with utilities that are required to protect the health and safety of a community" according to the local EM (Mitigation Assessment Team, 2012). These critical facilities include fire departments, police stations, hospitals, and waste water treatment plants among others. Since resiliency policy has been promoted when participants are included in the development of realistic climate visualizations (Sheppard et al., 2011) and FMs can make resiliency policy changes, including FMs in the development of storm impact models may promote storm resiliency actions. FMs will also likely participate in future storm-training simulations. Therefore, creating a method that gathers FMs concerns for incorporation FM's perceptions into storm impact models particularly useful.
We thus asked local EMs to select FMs to interview since FMs are highly informed about their facility (Mendonça & Wallace, 2006) and external resources their facilities rely on (Rinaldi, 2004). We sent selected FMs an invitation through email and called them over the phone to schedule a time to meet while simultaneously explaining the overall goal of the research Paul et al. (2018). Additional FMs were interviewed as the opportunities presented themselves (Patton & Appelbaum, 2003). For Westerly, this resulted in 13 FMs from 11 critical facilities including five fire departments, the police station, the dispatch center, the ambulance corps, the wastewater treatment plant, the water department, one electrical distribution substation and the department of public works (Figure 3). There are about 30 critical facilities in Westerly. We were unable to interview individuals from Westerly's hospital, school system, telephone networks, and natural gas facilities mostly because of the security threat posed by sharing the information.

Interview instrument design
In developing next steps to address the needs of FMs as they incorporate their concerns into storm impact models, ground truthing assumptions can help focus further research efforts (Becker, 2013). As a first step toward this goal, this exploratory survey gathered a wide range of FM's storm concerns for incorporation into storm impact models in order to identify the challenges and opportunities the method presented.

Interview process
In a few cases, an FM's colleagues joined the interviews, which may have contributed to the FMs comfort and is recommended for risk communication meetings (Chess et al., 1988). To begin interviews, we explained the study was meant to gather FM's storm concerns for incorporation into storm impact models (Chess et al., 1988;Paul et al., 2018). Then we explained the underlying storm models that the storm simulation utilized relied on (as recommended by White et al. (2010). We next used an illustration of a fictional CT for a petroleum facility triggered by storm surge from Hurricane Carol at 4:30 pm on August 5 th , 1954 overlaid on a modern day map at the Port of Providence as a thought prompt to show the interviewee how their responses could be used ( Figure   4). We told FMs their names would be kept confidential and quotes from their responses would not be identified and attributed to them individually, but might be discovered due to the limited number of critical facilities in Westerly.
Then we asked for the FM's immediate concerns to a storm. FMs were encouraged to consider storm impacts on other parts of the community (e.g. roads) that could affect their facility. We listened to FM's concerns, which is important for a risk communication spokesman (Chess et al., 1988), and then worked to identify the remaining CT components for each concern FMs identified ( Figure 5).

Figure 5. CT interview collection process.
When an FM identified a storm concern, we asked the FM to identify the location of concern on a navigable Google Map satellite (15 meters to 15 cm resolution) and/or street view using our laptop ( Figure 6). We also asked the FM to identify the concern increment using an open response threshold-identification method similar to the UKCIP and ICF methods (Monioudi et al., 2018;Wilson, 2015). For example, we asked, "What inundation level would cause the impact you mentioned?" Although FMs were encouraged to identify the storm forces that the underlying storm models used (rain, wind, storm surge, standing inundation and wave height) FMs elaborated on other weather concerns they had.
If FMs had trouble coming up with storm concerns, follow-up prompts were used to stimulate the conversation (see appendix I). Not all questions were covered in each interview, and the precise wording and order in which the questions came up were not constrained (Merriam, 1988) since interviews were focused on subjects that matched the interviewee's knowledge and points that the interviewee brought up (Lewis & Sheppard, 2006). The questions also gathered information on the FM's career experience and responsibilities at the facility. Interviews lasted between 1 and 2 hours.

Coding method
We digitally recorded and transcribed the interviews in full with the help of a hired transcription service (200 pages). In order to answer our three research questions we identified and analyzed all CT components mentioned by FM and major themes that occurred throughout the interviews using Excel and NVivo. We coded interviews line by line and identified themes in the data through an analytic induction method, a form of grounded theory described by (Ratcliff, 1994) as an iterative process.

Preparing CTs to be incorporated into storm impact models
We standardized CT components before incorporating them into the storm impact models. As mentioned earlier, CTs require five components for incorporation into storm impact models.
1. The concern -An asset or operation the FM perceives can be directly impacted by a storm force. For example, a generator that can be impacted by flooding.

The accurate location of concern -
The latitude and longitude of the asset or operation the FM believes can be directly impacted by storm forces. For example, a generator's location at 41.12345 N and -71.12345 W. (Stempel et al., 2018).
3. The modeled hazard -A storm force that can be quantified such as inundation depth or wind velocity that is modeled as part of a storm simulation (Stempel et al., 2018). 4. The increment -The storm force threshold that when exceeded at the location of concern triggers an impact. For example, winds above 100 mph or flooding above 1 foot. (Stempel et al., 2018).
5. The impact -The outcome if the storm force exceeds the increment at the location of concern. For example, the generator would short-circuit.
To standardize these components for inclusion into storm impact models, we wrote the impact component in similar terms the FM used during the interview. Also, to standardize increment responses, the following actions were taken ( Table 2).

Language used by FM Example Increment chosen Rationale
Threshold range "The impact would occur when the water reached 1 to 2 feet here."

foot
Choosing the lowest value in range makes the storm impact model output more conservative.
Hurricane category "The impact would occur when winds reached category 1."

mph
This is the sustained wind speed required to be considered a category 1. .

CHAPTER 4: RESULTS
The results below are themes identified while analyzing the interviews and the gathered CT components. These themes are referenced in the discussion to answer our three research questions.

RQ 1: How can existing methods for eliciting vulnerability data be adapted to gather FM's intra-facility level storm concerns for inclusion in storm impact models?
Our CT-gathering method identified 197 concerns from 13 FMs representing 11 critical facilities. We identified every CT by showing FMs potential uses of their responses through an illustration of a CT (Figure 4), asking for the FM's concerns using prompts (see appendix I) and then identifying remaining CT components on the FM's own terms ( Figure 5). We restated the FM's concern and specifically asked for each CT component for almost every CT. FMs guided us to almost every location of concern using the navigable Google Maps satellite view during the interview. In some cases, FMs located and marked the location of concern themselves on the Google Map. The Google Map's Street View allowed FMs to identify locations of concern that were hard to identify on the satellite view.

RQ 2: What challenges exist as researchers gather FM's concerns for incorporation
into storm impact models?

FMs could not identify modellable increments
74 of 197 CTs could not be incorporated into storm impact models because the increment was unknown, not explicitly asked for, or was not given in units the underlying storm model currently uses (Table 3). All but nine increments were coded as uncertain. This means the FM was willing to estimate the increment, but said they unsure of its accuracy. 40 of the consequence thresholds were coded as "unknown".
Increments were uncertain or unknown with regards to wind speed and inundation level because the FM knew a precise increment existed for the concern, but did not know it. One FM illustrated this dilemma when he said, "for an antenna to break, each one has a rated wind speed velocity--and you have to look up each and every antenna because there's multiple types of antennas, different manufacturers." Also, FMs had to consider if the equipment had been properly installed and mounted properly.
Only one increment was modellable for rainfall and wave height and no increment was modellable for storm surge because FM's do not know rainfall rates in inches per hour or storm surge velocities in any modellable increment (Table 3). One FM illustrated this when he said, "When it rains, it rains hard! That's all I know." Similarly, for storm surge, another FM explained that he would not drive his fire truck through water, "If you could see a decent current with any type of like a ripple to it." Many concerns and associated impacts were based on the FM's experiences with the Floods of 2010 and Hurricane Sandy. When asked for the increment that would cause the impact, FM's would ask the researcher for wind speed/storm surge levels of Hurricane Sandy and the rainfall amounts of Floods of 2010. FMs also tried to look up these increments online during the interview. One FM illustrated a commonly given unmodellable increment with the quote, "Whatever the 2010 floods were, how much rain we got [caused the impact]." Table 3. Total number of concerns, description of concern for each modeled hazard and coding classification of corresponding increment component for 13 Westerly FMs of 11 critical facilities.

FMs identified unmodellable location of concerns
FMs said that 40 of the 197 concerns could occur at many locations in Westerly and did not specify those locations. For example, one FM was concerned that a storm would flood fire hydrants and prevent his crew from reaching them. However, the FM did not know the locations of the fire hydrants that were at risk to flooding and therefore they could not be incorporated into the storm impact model. FMs gave the following concerns that had many locations and did not identify those locations so without further research could not be incorporated into storm impact models. immediately if water reaches them and then require a long-term chemical treatment process once the floods recede.

Storm impact model outputs FM's impacted concerns in chronological order
After incorporating the gathered CTs into a storm impact model that used a hypothetical storm, called Hurricane Rhody as the underlying storm simulation (Ginis, 2017) we found that 23 wind and 21 inundation CTs were triggered. The following illustrates and describes the 44 CTs triggered by Rhody in chronological order.

Figure 7 . Storm impact model for Westerly's critical facilities showing wind and inundation CTs triggered by simulated storm Rhody 30 minutes (top) and two hours (bottom) after Rhody's storm surge makes landfall. The red pin in front is
a CT triggered by one foot of flooding and represents a fire chief's concern that, "When it floods above a foot here, we can't reach the homes around the lighthouse." Yellow pins show where inundation blocks roads.
Hurricane Rhody's winds knock out Westerly's power and no emergency responder has communications, aside from cell phones and hand held radios. 24 to 48 hours later, Rhody's storm surge makes landfall. Within 30 minutes of landfall, the storm surge blocks a fire department from reaching sections of homes (Figure 7).
Within the next 30 minutes, Westerly's water distribution pipes may be broken and the water supply of the town may be contaminated. Between hour 1.5 and 2, the storm pushes water further inland and impacts the WWTP, which requires the plant to operate as a primary facility for 10 to 21 days and receive essential materials from another WWTP. Between hour 2 and 2.5 Westerly's main power station is impacted, and requires a mobile substation. Also, the water department requires two wells to shut down and chemically treated for 72 hours.

CHAPTER 5: DISCUSSION
This research gathered FM's intra-facility level concerns for incorporation into storm impact models and identified challenges and opportunities the process posed.
Identifying the challenges presented while gathering FM's intra-facility level concerns for incorporation into storm impact models contributes to answering Moser 2010's call for "applied research that links geographic information to practitioners concerns" and 3. The identification of improvements to storm impact models made by developing the models with FMs using a participatory approach.

How can existing methods for eliciting vulnerability data be adapted to gather FM's intra-facility level storm concerns for inclusion in storm impact models?
Participatory mapping using Google Maps and a set of exploratory open-ended questions was an effective way to gather FM's storm concerns for incorporation into storm impact models. Beginning this process by asking for the FM's storm concerns led to a list of concerns the FMs had already experienced. Many of the concerns mentioned at the beginning of interviews had been impacted by the Floods of 2010 and Hurricane Sandy. Once all CT components were gathered for these historical concerns, then the researcher could bring up concerns the FM had yet to experience to determine more CTs. This process may have gathered many CTs because it allowed the FM to guide the conversation and made the FM feel like the researcher was considering their concerns (Chess, Hance, & Sandman, 1988 Street View also helped identify the location of concern. Identifying the location of concern on a Google map made it easier to record FM's concerns and identify remaining CT components. For example, identifying locations of concern helped identify increments because we could explain to the FM that increments should be for the location the FM identified on the map. This was useful because FMs tended to identify inundation increments in feet above sea level instead of feet above ground level at the location of the concern.

What challenges exist when gathering FM's concerns for incorporation into storm impact models?
The most common challenges when gathering FM's concerns for incorporation into storm impact models are uncertainty of the increment, uncertainty of the location of concern and recording all CT components during the interview.

Challenge 1: The modellable increment identification challenge
Researchers conducting climate change vulnerability assessments have found that determining the quantified weather force that would cause a particular impact on an Indonesian community was, "considerably more challenging than [determining] qualitative descriptors like hotter, drier and rainier" (Gustafson, Cadena, & Hartman, 2018). Also, Shackley and Deanwood 2003 found stakeholders were reluctant to quantify climate forces and associated impacts because quantification "precisely defines the model scenario and is less likely to be correct than less accurate qualitative statements." This research found that stakeholder's uncertainty in identifying increments also extends to storms.
To help future researchers gather FM's concerns for incorporation into storm impact models we recommend researchers collect increments with FMs by asking a set of standardized closed questions with consistent anchors after an FM mentions a concern. For example, the researcher should ask, "Would the concern be impacted when water reached between 0 and 1 foot (anchor 1), 1 to 2 feet (anchor 2), or above 2 feet (anchor 3) at the location of concern?" Since many FMs referenced previous storms that impacted their facility to identify the increment, we recommend range of values, researchers should ask for the concern's make and model so the researcher can look up the increment after the interview. In order to promote effective policy through model building, iterative processes like this are key (White et al., 2010).

Challenge 2: The unknown locations of concerns challenge
20% of FM's concerns could be impacted at many locations. For example, FMs were concerned about power lines, roads and fire hydrants. Without identifying the exact latitude and longitudes of these concerns, it is not possible to incorporate them into storm impact models. To incorporate CTs with multiple, currently unknown locations into storm impact models we recommend the creation of datasets with these locations. Determining the locations of concern for sewage manhole covers, fire hydrants, telephone/power lines and the remaining concerns with many unknown locations was outside of the scope of this work. However, we found an existing data set for the locations of roads in Westerly, which allowed us to conduct a roadway low point analysis for Westerly to account for FM's road inundation concerns ( Figure 8).
When water reached above 8" at the low point on the road, the road was flagged as impassable since that is the depth some Westerly FMs are blocked. Similar analyses can be done for remaining CTs that have vague locations of concern once locations of concerns are gathered for those CTs.

Challenge 3: Not asking for CT components for all FM concerns during interviews
Although the method presented here gathered 197 concerns, researchers did not hear 34 concerns mentioned by the FMs during the interview and therefore did not ask for the remaining, required CT components.
To prevent researchers from missing concerns identified by FMs, we recommend marking each location of concern on the Google Map as the interviewee identifies the location and then filling out remaining components later in the interview. We recommend using a satellite map with more advanced labeling functions than Google Maps (e.g. Google Earth). This will make it easier for the interviewer to label all CT components directly on the map as the FM explains their concerns during the interview.

How are storm impact models improved by gathering FM's concerns through a participatory process?
Gathering FMs concerns for incorporation into storm impact models through a participatory approach improves storm impact models by increasing their accuracy and relevance to participants. Traditional DRR assessment outputs like HAZUS struggle to provide actionable data regarding relevant, specific local concerns for communities to use to prepare for disasters (Paul et al., 2018). Caution needs to be used when analyzing HAZUS hurricane damage outputs for particular facilities because results are based on average damages to similar facilities under similar circumstances (Vickery et al., 2006). Traditional DRR outputs like HAZUS also do not take into account intangible consequences of storms like losses of cultural assets (Messner & Meyer, 2006) even though these losses are what people normally bemoan the most after a flood (Green, Wierstra, Penning, & van der Veen, 1994). Therefore, including FMs concerns using a participatory process allows storm impact models to output more specific local concerns than relying on generalized damage curves and likely makes the models more credible, actionable and relevant to participating FMs.
As storm simulations increase in accuracy (Aerts et al., 2018), on the ground vulnerability information will need to be gathered with increasing precision using a participatory method to most effectively use these models. Gathering this information in a standardized way may eventually allow FMs to use storm impact models to predict and prepare for real storms in the days and hours leading up to landfall (Brecht, 2007;Ginis, 2017).

Limitations of this approach
Most increments identified by FM's were uncertain. However, the storm impact model this research created did not account for this uncertainty because we did not gather that information from FM's during interviews. When an FM said they were unsure of the increment, we asked the FM to identify a particular value. If the FM identified a range of values, we chose the lowest for incorporation into the storm impact model and ignored the rest of the range. Precisely quantifying the storm force at which an impact occurs makes the output appear more certain than it is (Shackley & Deanwood, 2003). When experts identify a point which a certain piece of infrastructure will fail, it is likely that the point has a good deal of uncertainty, which is not easily shown by the point (Cooke & Goossens, 2004). Future research that gauges the uncertainty of stakeholders increments will enable storm impact models to show uncertainty when developing visualizations of the data (Liu & Palen, 2010) and will make the models more useful to decision makers (Couclelis, 2003). Future research should be conducted to validate increments identified in this research before using them to justify DRR activities.

Conclusion
This paper presents an exploration of a methodology that gathers FM's rich concerns at the intra-facility level for incorporation into storm impact models. More than 100 concerns were gathered for incorporation into storm impact models using principals from participatory mapping and vulnerability assessment literature. After incorporating these concerns into storm impact models, we found the chronological order a hypothetical storm would impact those concerns. Gathering FM's concerns for inclusion into the storm impact models likely increases the relevance and credibility of the model's outputs to participating FMs. This research also identified challenges of gathering FM's concerns for incorporation into storm impact models and presents recommendations to overcome those challenges. This includes FM's uncertainty of specific storm force increments that would cause impacts. This research thus calls for an exploration as to why FM's are uncertain about increments and a way to identify increments using a closed-response method that accounts for FM's uncertainty. As storm simulations increase in accuracy and applicability, researchers will need to develop standardized methods to gather on-the-ground vulnerability information in a participatory manner to increase the relevance and credibility of storm impact models.
This research acts as a guiding step in the creation of such a method.
. for presentations at conferences, used in educational settings and may be published in scientific journals. Although no facility identifiers or detailed point data will be included (e.g., highly specific location or vulnerability data), locations may be recognizable to people who are familiar with the context depicted 3. CONSEQUENCE THRESHOLD COLLECTION This section enables the interviewee to identify consequence threshold components. However, in many cases the questions will require the interviewer to develop remaining components of consequence thresholds by furthering the interviewee's ideas (CONCERN, HAZARD, IMPACTS, and/or INCREMENTS).
Document driven questions 17. Does your facility have a written physical security plan/business continuity plan/that: a. identifies risks to a storm? b. evaluates the potential damage or loss to the facility due to a storm? c. identifies strategies to limit or control potential consequences of an incident caused by a storm? d. identifies storm response and recovery procedures for life safety, human resources, core operations, information technology, and other necessary organizational functions? 18. Does your facility have operational manuals that specify storm conditions for when your facility is unable to operate? 19. Do industry guidelines (federal guidelines/design specifications) specify storm conditions that your facility can withstand? 20. Has your facility conducted post storm assessments that identified consequence threshold components?
Miscellaneous 21. How have storms impacted facilities similar to yours? 22. Do you think facilities of your type have similar impact ranges? 23. If you could change or add one thing about your facility to prepare it for a storm what would it be? 24. Do you know the FMs of the facilities that your facility relies on and would be willing to answer the previous questions? a. If yes, can you connect me with them? 25. Did you prepare for these questions after our first telephone discussion? 26. Of the impacts we have determined today, which ones are most important to you? 27. Is there anything else you would like to add?

Interviewee background
28. To help me analyze my research, I would like to know about your work as a facility manager. 29. How many years of experience do you have preparing the facility you currently work at for storms? 30. Do you create, execute or contribute to plans to keep your facility functioning after disruptive events such as a storm? If yes, please explain what this work entails. 31. Do you create, execute or contribute to resiliency related activities? These activities include conducting impact analyses or risk assessments that take into account your facilities significant assets (assets essential to the function of the facility), functions (e.g., IT systems), personnel, and supply chains. If yes, please explain what this work entails.