Tipultech logo

Pilot Study

Sarah Ogilvie

A pilot study was conducted to test the utility of the 12 frame types derived in the previous section. The aim in this study was to determine whether the proposed set of frame types adequately captured those used by negotiators. To confirm these findings, the typology was then applied to the data collected for the main study (45 negotiations). Using the data from these 45 negotiations, the inter-rater reliability of the typology was then assessed.

For the pilot study, negotiation data were selected from 10 negotiations conducted previously with a sample of 20 Mature-Age University students. The data used in this analysis were derived from negotiations that used the same type of transactional negotiation task as used in the main study. Each videotaped negotiation involved two MBA students. These students had been asked to negotiate features of an employment contract as part of a work requirement. Each negotiation was transcribed from a videotaped recording. One 5 minute excerpt was then randomly extracted from each of the 10 negotiations. Each speaking turn in these five minute excerpts was then coded using the frame typology. For speaking turns in which there were multiple different frames in use, the dominant frame was identified. The number of times each of the frames was used across the ten 5 minute negotiation excerpts was calculated and displayed in Table 2.

Table 2

Frequency and Proportion of Conflict Frame Employment Across Ten 5 Minute Negotiation Excerpts
Frame Type Frequency Proportion (%)
1. Cooperative Aspiration 22 6.5
2. Competitive Aspiration 29 8.4
3. Cooperative Outcome 66 19.2
4. Competitive Outcome 77 22.6
5. Substantive Issues 40 11.7
6. Relational Issues 3 0.9
7. Positive Characterisation 4 1.2
8. Negative Characterisation 5 1.6
9. Inclusive Process 9 2.5
10. Non-inclusive Process 5 1.5
11. Values-based Justification 7 1.9
12. Power/Rights-based Justification 31 9.3
13. Other 46 13.4
  344 100

A small proportion of utterances (13.4%) were not represented by the set of codes. These utterances, which were classified as other, were inspected and found either to be clarifiers (e.g., "You said $80,000?") or irrelevant statements (e.g., "What was that noise?"). Because irrelevant statements did not relate to the negotiation, they were excluded from the frames of interest. Similarly, clarifying statements were not of interest in the current study because the focus was upon assessing the way that information was framed and reframed and not upon the way that information was checked for understanding between the negotiating parties.

The only frame that was used during less than 1% of the speaking turns was the relational issues frame. This may have occurred because the negotiation task was a transactional rather than a relationship-oriented negotiation task (Thompson, 1998), and so did not prompt negotiators to frame information in terms of their relationship with the other party. Four other frames were found to occur during fewer than 2% of the turns: the positive characterisation frame, the negative characterisation frame, the non-inclusive process frame and the values-based justification frame. These five frames were retained for coding data in the main study.

Confirmation of Pilot Study Results Using Main Study Data

The set of 12 frames appeared to capture adequately the nature of the communication between parties, because only 13.4% of frames were not captured by the set of utterances included in the typology. However, to ensure the robustness of the frame typology, this typology was subjected to further validation using the data obtained from the 45 negotiations conducted for the main study.

In the main study, each videotaped negotiation involved two First Year Psychology students who participated in the experiment as part of a course requirement. A total of 90 participants (45 negotiations) were used in this stage. Each negotiation was transcribed from a videotaped recording, after which each uninterrupted speaking turn was coded using the set of 12 frames comprising the frame typology and the other frame category. The occurrence of each of the 12 frame types in the 45 negotiations is listed in Table 3.

Table 3

Frequency and Proportion of Conflict Frames in 45 Negotiations
Frame Type Frequency Proportion (%)
1. Cooperative Aspiration 369 8.2
2. Competitive Aspiration 319 7.1
3. Cooperative Outcome 1016 22.6
4. Competitive Outcome 1110 24.7
5. Substantive Issues 360 8.0
6. Relational Issues 13 0.3
7. Positive Characterisation 45 1.0
8. Negative Characterisation 67 1.5
9. Inclusive Process 274 6.1
10. Non-inclusive Process 108 2.4
11. Values-based Justification 63 1.4
12. Power/Rights-based Justification 301 6.7
13. Other 450 10.0
  4495 100

To reduce the frame typology to a smaller set, frames occurring during less than 2% of the speaking turns across the two stages were either excluded or combined with other frames on the basis of their definitional similarity. The frames that were used less than 2% of the time were the relational issues frame, the positive characterisation frame, the negative characterisation frame and the values-based justification frame. This finding was largely consistent with the results of the pilot study and, hence, these four frames were considered for exclusion or for combining with remaining frame categories.

After inspecting the definitions of these four frames within the context of the complete set of frames, and pairs of frames that were related but, even when combined, did not exceed the 2% criterion required for the typology were not integrated. Likewise, pairs of frames were not combined if they corresponded to opposite ends of a single dimension. No recombinations of these frames fulfilled the criteria that defined acceptable clusters. Therefore, these four frames, the relational, the positive and negative characterisation frames and the values-based justification frames, were combined into the other category. Accordingly, the proportion of communication utterances falling outside of the frame typology rose from 10% to 14.2%, and the total number of frames in the typology diminished from 12 to 8.

A correspondence analysis was then conducted on the reduced set of eight frames. This type of analysis is typically used to determine whether there are relationships that exist between categorical variables (Greenacre, 1984). In the current study, correspondence analysis was used to determine whether any of the eight frames were correlated, and on the basis of similarity in use could be combined (Figure 1). The results indicated that two frames were similarly used by negotiators: the competitive aspiration frame and the power/rights-based justification frame. Although these two frames share some similarity in their orientation, they nonetheless appeal to markedly different areas: Where aspirations refer more to the underlying needs and concerns of the individual and in this sense can be perceived as justifications made on the basis of more individually-relevant internal factors, justifications of claims in terms of power or rights involve drawing upon more external factors (Fisher and Ury, 1981). For this reason, these two frames were not combined in the main analysis.

Correspondence analysis results for eight conflict frame types.

Figure 1. Correspondence analysis results for eight conflict frame types. The final set of frames used in the main study was therefore:
1. Cooperative Aspiration Frame.
2. Competitive Aspiration Frame.
3. Cooperative Outcome Frame.
4. Competitive Outcome Frame.
5. Substantive Issues Frame.
6. Inclusive Process Frame.
7. Non-inclusive Process Frame.
8. Power/Rights-based Justification Frame.

Inter-rater Reliability

To test the inter-rater reliability of the final set of eight conflict frame types amongst the 45 negotiations collected for the main study, 15 negotiation transcripts were randomly selected and analysed. Two raters, both blind to the aims of the study, coded each uninterrupted speaking turn. Because each speaking turn may have included more than one frame, the raters were instructed to code for the dominant frame in each case. The degree of agreement between raters for each of the frame types was measured using Cohen's Kappa (Table 4). Across the eight frames, the average reliability was 81%. All Kappa statistics indicated a substantial to high level of agreement between raters (Landis & Koch, 1977). Any disagreements between coders were discussed and recoded on the basis of these discussions.

Table 4

Inter-rater Reliability for Each of the Eight Conflict Frame Types
Frame Type Reliability Index (Cohen's kappa)
1. Cooperative Aspiration 0.82
2. Competitive Aspiration 0.85
3. Cooperative Outcome 0.81
4. Competitive Outcome 0.76
5. Substantive Issues 0.85
6. Inclusive Process Management 0.83
7. Non-inclusive Process Management 0.76
8. Power/Rights-based Justification 0.85

<< >>