Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

Using Computer-Aided Content Analysis to Map a Research Domain: A Case Study of Institutional Legitimacy in Postconflict East Timor

Using Computer-Aided Content Analysis to Map a Research Domain: A Case Study of Institutional... Content analysis is a powerful tool for investigating attitudes expressed in naturally occurring language data. It is a useful tool to help researchers develop an understanding of a specific research field through identifying how particular issues or topics have been conceptualized or where fieldwork can be limited or prohibitive. This is especially true for research on postconflict reconstruction, where large-scale quantitative surveying or metareviews of the literature can be prohibitive. The present study provides a case study of how a particular content analysis software program—Leximancer—was used to map factors associated with institutional legitimacy in postconflict societies. The case of Timor-Leste is used as an example. We examine texts at three levels of discourse: at the academic, official, and primary levels. Results indicate differing perspectives on legitimacy at each level of discourse. This article offers a snapshot of a potential method for understanding how particular topics are conceptualized within a specific research field and can thus help in the development of evaluation methods or data collection instruments. Keywords legitimacy, content analysis, research methods, postconflict societies, social identity such as ethnographic fieldwork or qualitative interviewing Introduction does provide opportunities to gauge local insights, they can Although the legal and international legitimacy of peace- suffer from small sample sizes and hence lack external valid- building missions is regarded as one of the most important ity. assets of efforts to rebuild failed states (Barnet & Zurcher, The aim of this article is to test a potential alternative 2009; Brinkerhoff, 2005; Chesterman, Ignatieff, & Thakur, approach (i.e., content analysis) that could be used as a 2005; Lemay-Hebert, 2009; Mersiades, 2005; Roberts, “complement” to other forms of data collection or as a stand- 2008; United Nations, 2008), there is a lack of research alone method. Content analysis examines the conceptual examining factors that influence domestic legitimacy among structure of existing opinions expressed in text-based sources local populations. Domestic legitimacy refers to whether of information. We recognize that such an approach has its peacebuilding processes (here we include statebuilding as limitations, particularly in postconflict contexts where low well) are accepted and supported by local groups. It incorpo- literacy rates, limited access to the Internet, and an embry- rates beliefs and attitudes and has a behavioral outcome, in onic print media mean local opinions can be underpresented that it impacts on people’s willingness to cooperate with in different text-based data sources. However it can provide institutional authorities. Currently there is no defining meth- a viable starting point for understanding how particular odology for assessing the attitudes of local groups and stake- issues or concepts are understood or conceived, which then holders toward reconstructed institutions and the agents provides useful data to prepare fieldwork methods or under- responsible for rebuilding them. Evaluation methods may take closer analysis. include population-based studies of public attitudes in post- University of Queensland, St Lucia, Australia conflict societies, though achieving a reliable nationwide sample in a postwar country is logistically difficult. By the Corresponding Author: time results are collected and analyzed, the shifts in political Adrian Cherney, School of Social Science, University of Queensland, St circumstances may have rendered the surveyed public opin- Lucia 4072, Australia Email: a.cherney@uq.edu.au ion irrelevant. Likewise, although in-country data collection 2 SAGE Open To gain insight into the issue of the legitimacy of recon- Saldanha, 2008). For example, the 2007 elections produced structed institutions, not only are local attitudes measured a majority vote for the FRETILIN (Frente Revolucionária via analysis of online forums and blogs but also other groups do Timor Leste Independente) political party (29% of votes; with varied involvement in peacebuilding processes are Higashi, 2009), yet the government was eventually formed included in our analysis: that is, “officials” (including local by a coalition government lead by popular Prime Minister government, United Nations, and nongovernmental organi- Xanana Gusmao. FRETILIN denounced the coalition as zation [NGO] text sources) and “academics” (journal arti- unconstitutional and illegitimate, with the legitimacy of the cles published by experts in the field). These three sources elected government remaining a dividing issue for locals of information can provide differing perspectives on the (Higashi, 2009). Timor-Leste is a useful study for assessing legitimacy of state and international institutions involved in institutional legitimacy after peacebuilding and reconstruc- postconflict reconstruction and in the process, allow one to tion efforts due to the mixed outcomes of the reconstruction gain insight into the rationale for those opinions. Leximancer, process, with it struggling to maintain basic legitimacy in a text analytics program, is used to facilitate content analy- the eyes of the local population. sis, with it helping to visualize the meaning of content in texts by extracting concepts, and then graphically mapping Theoretical Framework relationships between concepts (the use of Leximancer for content analysis will be explained in the “Method” section). Effective content analysis relies on a theoretical framework This study is exploratory in nature and we recognize that through which to collate textual data and categorize the there are limitations in our approach. Our aim is to demon- emergent ideas. As stated above, the focal point of the strate a specific methodology and how it is relevant to stud- content analysis in this article is domestic legitimacy, that ies on postconflict reconstruction and outline a replicable is, the extent to which Timorese perceive their authorities, approach for achieving this. institutions, and social arrangements to be appropriate, Timor-Leste has been chosen as a case study for this proper, and just (Brinkerhoff, 2005; Tyler, 2006), and how method due to its history of violent conflict, institutional these perceptions compare with those expressed by officials damage, and subsequent reconstruction. Indonesian forces and academics. M. Weber (1964) links legitimacy to a invaded East Timor in 1975 and occupied the country for 25 population’s willingness to comply with a system of rule, years, and Indonesian rule throughout this time was charac- and identifies three sources of legitimate authority: ratio- terized as oppressive, with the army silencing dissent. nal-legal authority, which describes a belief in the legality Regardless, opposition to the Indonesians increased in the of the rule and the right of the leaders to issue commands; 1980s, and in 1999, under strong local and international traditional authority, which describes a belief in traditional pressure, the Indonesian government announced it would systems of rule based on history; and charismatic authority, hold a referendum on granting independence to East Timor. a devotion to an individual leader with whom the populace While the independence vote was overwhelmingly passed, feel a personal connection, legitimizing the authority Indonesian soldiers and prointegration militia embarked on of their decisions (M. Weber, 1964). The perception of a campaign of violence, resulting in the death of 1,400 legitimacy is essential for social stability—when it exists in Timorese, the destruction of basic infrastructure, and the the thinking of people and groups, it leads them to defer to displacement of approximately 300,000 people. One month authorities, institutions, and social arrangements as right later, an international mission called INTERFET (Inter- and proper (Tyler, 2006). Legitimacy promotes voluntary national Force for East Timor), led by Australian troops, was cooperation between people and institutional authorities deployed to stabilize the country. This administration was fol- and leads citizens to accept decisions and rules, and also lowed by the establishment of UNTAET (United Nations encompasses the consent and acceptance of local actors to Transitional Authority in East Timor), which oversaw international interventions. This has an important bearing all aspects of governance of Timor-Leste. Democratic elec- on local popular support for rebuilt institutional authorities tions were held in 2002 and UNTAET was disbanded. such as the police, judiciary, and government (Brinkerhoff, Although the security situation in Timor-Leste remains 2005; Chesterman, 2004, 2007; Paris & Sisk, 2009; United fragile (including unrest in 2006 that led to the deaths of Nations, 2008). nearly 40 people and an attempted assassination of President Studies of domestic legitimacy have provided varied Jose Ramos-Horta in 2008), a recent UN Human Rights insights into the micro-level factors that influence the success Report praised improvements, especially in reconstructed of peacebuilding and statebuilding programs “on the ground” security and justice institutions (United Nations Integrated (e.g., Harmer & Frith, 2009; Mersiades, 2005; Rubinstein, Mission in Timor-Leste [UNMIT], 2010). Nonetheless, 2008). One factor contributing to perceptions of legitimacy is research has drawn attention to the problems Timor-Leste the notion of voice. Voice refers to the opportunity for groups has faced in establishing the legitimacy of its transitional to comment on or inform an authority’s decision, allowing authorities as well as the new democratic government them to have some level of input into processes that affect that followed (Croissant & Schacter, 2008; Olssen, 2009; them (Folger, 1977). Voice provides for some level of local Fisk et al. 3 control and is premised on the concept of local accountability The variables of voice and social identity have been and participation in the reconstruction process. This is essen- adopted for theoretical and methodological reasons. First, tial to ensuring that reconstruction efforts are perceived as we wish to employ a flexible theoretical framework to guide being aimed at meeting local needs and avoids the problem of the analysis of the qualitative data but still allow these data them simply being perceived as a vehicle for local elites and to speak for themselves by giving space to new emerging international or corporate interests (Chesterman, 2007; themes. Second, given the data comprise a large collection of Harmer & Frith, 2009; Richmond & Franks, 2008). In short, text collected from the Internet, there was a substantial voice builds trust within and across social groups, and is one amount of material that was not relevant to our general of the biggest predictors of whether people perceive pro- research question. Our theoretical framework helped to filter cesses to be just and in their best interests (Folger, 1977). out this irrelevant material. However, it needs to be stressed Hence, we have selected voice as one variable through which that the concepts and themes discovered within the data are to study domestic legitimacy. emergent, revealed through the use of Leximancer, a text Past research also indicates other factors that have the analytics software program. Although this software and the potential to influence perceptions of legitimacy. For example, analytical method underpinning it (described in the next sec- Rubinstein (2008) draws on a cultural perspective and argues tion) has been demonstrated to be a reliable and valid method that any explanatory framework aimed at conceptualizing of content analysis (e.g., see Smith & Humphreys, 2006), the local-oriented legitimacy needs to take account of the social interpretation of the results Leximancer produces still stratifications among local populations that generate bound- requires a theoretical lens through which to comprehend the aries along cultural and ethnic lines. These stratifications various patterns revealed in our data sources. As new sources make the engagement and inclusion of various local actors of data are added to Leximancer and different techniques of complicated and contested but is essential to achieving a sta- exploring the data are adopted, the content of concepts and ble peace. Social identity theory (Tajfel & Turner, 1979; themes revealed in the data can shift. A theoretical frame- Turner, 1991; see Hornsey, 2008, for a recent review) pro- work is essential to identifying generalizable results and vides further insight, proposing that people are quick to cate- interpreting instances that do not fit with consistent trends gorize the world into in-groups and out-groups, based on emerging from the data. religion, ethnicity, age, culture, socioeconomic status, and political beliefs. Furthermore, these group allegiances pro- Method duce systematic biases in perception and behavior (e.g., Brewer, 1979; Cohen, 2003; Hewstone, Rubin, & Willis, Content Analysis 2002; Tajfel & Turner, 1979). One consequence of these biases is that group membership can act as a heuristic that Content analysis is a research method that uses a set of tells people who can be trusted and who cannot, independent procedures to draw valid inferences from texts (R. P. Weber, of any history of interpersonal exchange. It is assumed that 1990). Such inferences can relate to the attitudinal and in-group members can be trusted because they are motivated behavioral responses of organizations or people to particu- to protect the common good. Out-group members, however, lar issues (Krippendorff, 2004; R. P. Weber, 1990). Content are instinctively mistrusted because it is assumed that they analysis techniques determine the presence of words or are motivated to protect their own constituency at one’s own concepts in collections of textual documents and can be expense (Foddy, Platow, & Yamagishi, 2009; Hornsey & used to break large amounts of materials into manageable Imani, 2004). These biases could potentially have profound categories (Leximancer Manual, 2010; Stockwell, Colomb, effects on the extent to which processes central to statebuild- Smith, & Wiles, 2009; Krippendorff, 2004). It can be used ing are seen as legitimate. This is because citizens can draw to understand the conceptual structure of a set of documents assumptions about which groups are responsible for rebuild- and identify the most important occurring themes evident ing institutions (i.e., who is “pulling the strings”). For exam- within text-based data relating to a particular domain (e.g., ple, a local population may find themselves speculating about political, social, or economic). who is “pulling the strings” when it comes to developing a constitution or a parliament: Is it the local populace, the Leximancer United States, the United Nations, the European Union, spoil- ers allied to the government, big business, or a complicated Leximancer is a content analysis computer software that combination of all of these? The answer to these questions identifies core concepts within textual data (conceptual (which can also be based on fact, myth, intuition, or conspir- analysis) and clarifies the properties of these concepts and acy) would be a key determinant of whether people see rebuilt how they are related (relational analysis). Leximancer identi- institutions as valid and legitimate. Hence, the second contex- fies what concepts exist in a set of texts, allowing concepts to tual factor that provides a focus for this study is social be automatically coded. For example, Leximancer can take a identity. set of 1,000 documents, containing millions of words, and 4 SAGE Open produce a map containing the 50 most important concepts assessments and official committee reports. Primary within that text, along with their relationships with each sources encompassed any text representing the viewpoint other. The overall advantage of Leximancer is that it analyzes of locals. To retrieve this information, a variety of sources textual data in a more reliable fashion and produces visual were used: online discussion forums, social media, per- representations that aid interpretation (Smith, 2003; Smith & sonal blogs, local online media (including comments sec- Humphreys, 2006; Travaglia, Westbrook, & Braithwaite, tions), and transcripts from investigative journalism 2009). programs in which locals were interviewed. Many of these Leximancer concepts and themes. A concept in Leximancer sources included texts translated from Portuguese and is a set of words that travel together (co-occur) in the text. Tetun into English. The aim of gathering primary text was Leximancer identifies relationships by the frequency with to access the perceptions of citizens: their thoughts, beliefs, which words occur together in a block of text and the fre- and attitudes to the conflict itself and subsequent recon- quency with which they occur separately. This means that struction. Nonetheless, “local” data in this study does not concepts themselves are emergent, rather than selected by reflect the entirety of Timorese society, given its reliance the author. Concepts that co-occur often within the same on Internet access and literacy levels. two-sentence block attract one another strongly and are clus- tered in the visual map produced by Leximancer. Similar Analysis Process concepts tend to settle together in close proximity, which aids interpretation by the researcher. Leximancer “themes,” Analyzing all text sources in Leximancer created an initial however, are clusters of concepts, denoted by circles on the exploratory figure. After perusing the resulting visual maps (see figures provided). Concepts that are not close figure, concepts deemed to be irrelevant were removed. We enough to any existing theme will begin their own theme. excluded common function words (and, not, etc.) that The size of themes can be controlled by the software user comprise a standard set of excluded words in Leximancer. and set to the level of explanatory detail desired. Each theme We also removed general terms (such as area, total, section) is named after the most prominent concept in that group, that were used commonly in the text but that did not con- which is also indicated by the largest dot in the theme tribute meaning to the concept maps. In addition, we cluster. merged most singular and plural words; the words culture and cultures were eventually merged because they were closely associated. It was particularly necessary to delete Data Sources for the Current Study concepts in the primary figure, as the content was down- A variety of sources were used for analysis. A total of 1,653 loaded in its entirety as HTML and therefore many tags, documents were included in the analysis, distributed equally headlines, and extraneous material were also included as between each source. Postcollection, all literature was orga- concepts. The content of these concepts were carefully nized by source (see appendix for source lists). The purpose examined to determine with certainty that a concept was of this organization by source was to allow for subsequent the result of irrelevant HTML material and not part of per- comparisons from differing points of view. For this reason, tinent text. After deleting these concepts, the analysis was the three categories of textual sources were deemed aca- run again. demic, official, and primary, to reflect the differing view- From this point, it was possible to assemble a list of points of those observing, leading, and personally affected words that could be used as a starting point for Leximancer by statebuilding. This is not to say that these groups are analysis, termed seed words. This technique is known as mutually exclusive, nor entirely homogeneous: for example, “profiling,” and is a means of defining the scope of the con- UN officials may publish in academia, “local” sources may tent analysis to include only text data related to the notion be politicized elites, and government and UN sources may of legitimacy. Given their diverse origin within the aca- have differing viewpoints within the same text. But for the demic literature, the concepts of interest were rarely purposes of analysis, these categories represent three impor- referred to explicitly across the three key categories of text tant populations who have different levels of involvement in, sources. Hence, it was necessary to manually develop lists and views of, statebuilding. of associated terms relating to the concepts of legitimacy, Academic literature was retrieved primarily from refer- voice, and social identity that could then be entered into eed political and social science journals (published between Leximancer. For example, the notion of voice was identi- 2000 and 2010). Official literature comprised text produced fied as associated with ownership and participation, terms by those responsible for delivering peacebuilding programs common to the field of peacebuilding (see Chesterman, in Timor-Leste. This included UN official documentation, 2007; Harmer & Frith, 2009). The denial of voice was iden- including resolutions, mandates, and progress reports. It tified as related to the opposite of these terms: for example, also included material sourced from government agencies dispossession and exclusion. Discussions in the peace- and NGOs, from specific project outlines to country building literature concerning ethnic identity and class Fisk et al. 5 Figure 1. East Timor academic legitimacy profile: Main themes Figure 2. East Timor academic legitimacy profile: provided terms commonly associated with notions of social “Reconstruction” theme identity. These terms were also entered into Leximancer and used to construct concept figures. 3./east timor academic/ Results By the early months of 2000, a growing crisis of Academic Sources legitimacy had begun to develop, catalyzing what appeared to be a radical reorientation in UNTAET’s The academic legitimacy profile figure indicated that differ- approach to political authority and participation. On ent institutions requiring rebuilding after conflict were given May 30, UNTAET announced its intention to move well-balanced coverage in the academic literature when toward a period of “co-government” prior to the trans- discussing perceptions of legitimacy (see Figure 1). The fer of full authority to the East Timorese. theme of “government” emerged as central on the visual figure, surrounded by overlapping themes encompassing 6./east timor academic/ other institutions: “health,” “economic,” “security,” and “police.” The legitimacy crisis that resulted from UNTAET’s As the concept visibility in Leximancer was increased, early paralysis confirms the observation by several the central theme that grouped concepts together relating to analysts that the first weeks of a mission can be crucial legitimacy was “government.” Among the top-ranking con- to its ultimate success. cepts that were identified as occurring with government were those connected to issues of voice, for example, “participa- It is interesting to note that UNTAET is the main institu- tion” and “include,” indicating voice as an important issue tion identified in the co-occurrence of legitimacy and crisis. identified in the academic literature. Also appearing in the This runs counter to previous research suggesting that the East Timor legitimacy profile were characteristics or proper- United Nations benefits from a level of legitimacy beyond ties of legitimate institutions, in accordance with academic that of unilateral and even multilateral foreign agents, with theory and research: “functioning,” “appropriate,” “respon- UNTAET enjoying high levels of support among Timorese sible,” “accountable,” and “democratic.” Closely related was people (Higashi, 2009). Given that UNTAET is the most a “reconstruction” theme, in which legitimacy was a central ambitious transitional administration in the history of the concept (see Figure 2). United Nations, we anticipate polarized reactions, and these A query on the legitimacy concept revealed a negative criticisms leveled by academics will also be examined in the co-occurring concept: “crisis.” Text retrieved from original context of the results from the official and local sources. An text sources showed that in the perception of the academic investigation of the notion of “local” voice and participation community, the erosion of the legitimacy of UNTAET, par- resulted in a “knowledge pathway” between “local” and ticularly in its governance role, had created instability in “consultation.” A pathway is a Leximancer feature that East Timor in the period of 2000-2002. Below are some describes a relationship between two concepts. A start con- example quotes from the “legitimacy” and “crisis” query: cept is selected, followed by an end concept, and the links 6 SAGE Open Figure 3. Knowledge pathway between “local” and “consultation” concepts, academic sources between them are illustrated. The relationships between element of postconflict reconstruction and is regarded as tied these concepts are best thought of as correlations, with text to the legitimacy of agencies directing the reconstruction segments describing the relationship between each concept processes and the emerging institutions allocated responsi- in the pathway and the next (for further information, see bility for governance. Implicit in these academic conceptual- Leximancer Manual, 2010). Figure 3 charts the links between izations are notions of active participation in the peacebuilding each step as well as provides illustrative quotes. process, but a central question is whether such forms of We can observe from this result that in the academic lit- active voice have been achieved to the satisfaction of local erature, voice is explicitly linked to notions of legitimacy, populations. This is essential to whether local groups will particularly in the context of civil society, capacity build- voluntarily cooperate and support the activities of agencies ing, and good governance. Also in the top 10 concepts co- involved in peacebuilding missions and defer to their occurring with legitimacy are the concepts of “local” and authority. Social identity processes are alluded to in these “power.” academic sources, though not explicitly linked with percep- Summary of academic texts. From the academic legitimacy tions of foreign forces in Timor-Leste. These initial results profiles, the notion of “voice” is considered an essential will be further explored in the official and local texts. Fisk et al. 7 Figure 4. East Timor official legitimacy profile: Main themes Figure 5. East Timor official legitimacy profile: “System” theme Official Sources In the legitimacy profile of official sources, the representa- tion of institutions was less specific than in the academic Overall, results indicated that improving voice (i.e., par- literature (Figure 4). The main themes evident in this litera- ticipation) is identified as a key aim of reconstruction by the ture were “government” and “system,” which between them majority of government, United Nations, and NGO sources. thematically grouped the majority of concepts to appear on However, the above quote illustrates an important problem this figure. with statebuilding: the perception that government attempts The “system” theme comprised the majority of concepts to increase voice are aimed at appeasing international stake- and called for more in-depth analysis (see Figure 5). As in holders, rather than underpinned by a sincere and good- the academic figures, it illustrated a variety of concepts asso- willed attempt to actually encourage local participation. The ciated with legitimacy that are particularly applicable to official sources also contained assessments of attempts to notions of voice (e.g., “access,” “consultation,” enhance local participation through donor supported pro- “cooperation”). grams, for example, CEP—Community Empowerment and Building participation among locals was stated as an aim Local Governance Project—which is administered by the of most United Nations, NGO, and government policy docu- World Bank and the Asian Development Bank. One of the ments relating to statebuilding in East Timor. “Participation” aims of CEP was to empower Timorese through the creation was 1 of the top 10 concepts associated with “institutions” in of village and subdistrict councils to make funding the official literature, with a 16% likelihood that the 2 con- decisions supporting the local development of projects. cepts would co-occur in the same text segment. Also the top However, in its attempt to empower locals, CEP may have 10 concepts co-occurring with “institutions” were related had the opposite affect, as indicated through the following concepts of “involved” and “consultation”—reflecting dif- quote: ferent elements of voice. Figure 6 below illustrates the sec- tion of the map where these concepts were located. 7 ./east timor official/ Figure 6 reveals relationships between “voice” concepts in the official sources. Voice is referred to in the context of CEP’s attempt to bypass preexisting governance struc- institutions, co-occurring with concepts such as involved, tures with a new Council structure was received consultation, and participation. Also within this cluster is the ambiguously by communities and in many cases, gen- concept “responsibility,” wherein official sources discuss erated local conflict that inhibited the project. Instead, empowerment through involving local institutions. For the CEP model should have adopted the positive fea- example, the following quote from a local NGO illustrates tures of Timorese local authorities and challenged its this point: regressive features through continuous dialogue. 24./east timor official Another key issue to emerge related to rebuilding legiti- macy in the official literature was holding appropriate inves- We hope that this time the government will put more tigations into human rights violators as part of transitional efforts into making this consultation a true beginning justice processes (see Figure 7). for popular participation in Timor-Leste and not just Within the official literature there were a number of pro- use it to please international institutions. cess-related issues identified as important to institutional 8 SAGE Open Figure 6. Official map section containing concepts “institutions,” “participation,” “involved,” and “consultation” Figure 7. Official sources: “International” theme containing human rights–related concepts legitimacy, one being the necessity for commissions or trials and legal experts relating to the perceived legitimacy of into human rights violators to meet international standards Indonesia–Timor Leste Commission of Truth and Friendship and principles of fairness in the eyes of the general popula- (2005-2008), which investigated human rights abuses by tion. The official figures revealed some doubts among NGOs Indonesia during the occupation of Timor-Leste: Fisk et al. 9 15./east timor official/ While various governments on friendly terms with Jakarta applauded the long-stalled beginning of the court, human rights advocates have been far more critical. There are too many loopholes that could pre- vent the effectiveness of the trial, such as the skill of the judges, the law, as well as the independency of judges from elements of power. There is also some evidence that this perceived lack of legitimacy is related to Indonesian interests dominating the Commission: 19./east timor official/ The trials have failed fundamentally in fulfilling the “truth function,” a central part of the mandate of Figure 8. East Timor primary legitimacy profile: Main themes human rights and war crimes tribunals. Indeed, by clinging to a version of the violence in East Timor that Primary Sources is accepted nowhere outside of Indonesia (and rejected there by many), the trials have lost a unique opportu- The first figure resulting from the primary sources showed a nity to set the historical record straight, inform the contrast to the academic and official figures (Figure 8), with Indonesian public of the accountability of their institu- an emphasis on more humanitarian concerns: for example, tions for the gross human rights violations perpetrated “culture,” “rights,” “people,” “group,” “agree.” in their name, and provide a basis for reconciliation. Results from the primary texts indicated that for the Timorese people, a sense of transitional justice was of great This provides preliminary support for the social identity concern. Close inspection of the main themes in the primary hypothesis, with the motives and outcomes of the Commission literature showed that human rights considerations were cen- being innately distrusted and disputed by local Timorese due tral to perceptions of legitimacy and connected to almost all to suspicion over who is in control of the process. other issues in the profile. Summary: Official sources. Concepts relating to voice (i.e., When the theme size was decreased to reveal smaller participation and consultation) were closely bound to notions clustered themes, variations on Xanana Gusmao’s name of legitimacy in the official profiles. In some cases, govern- appeared on the primary figure within the people theme, very ment attempts to run community programs aimed at increas- close to a “leadership” concept (see Figure 9). Xanana ing voice were viewed as attempts to appease international Gusmao is a central figure in Timorese politics, the previous stakeholders, rather than a genuine attempt to use local opin- leader of the armed resistance turned first head of state. ion to inform policy implementation. In this case, the con- Interestingly, his name did not appear on either the academic cept of voice is tied to social identity processes, with locals or official legitimacy profiles. His importance to the distrusting institutions due to perceptions that they are influ- Timorese people may reflect a source of legitimacy that Max enced by outsiders and reflect foreign interests. This points Weber (1964) calls “charismatic authority,” wherein legiti- to one of the key challenges peacebuilding missions face, in macy for a political party or movement is gained through the that while efforts might be made to provide opportunities for power of an individual’s personality. The following quotes locals to have voice through processes of consultation, if emphasize the important legitimating role of Gusmao: those strategies do not translate into actual action that reflect the preferences of local populations—compared with donor 49./east timor primary/global voices online (4)~1. agencies—then the process and the outcomes will be viewed html/1/1_69 with suspicion and loose domestic legitimacy. Official sources also emphasize the path to legitimacy through tran- It’s fair also to hope that the national hero Xanana sitional justice and forgiveness mechanisms such as the Gusmão makes the best use of his well known cha- Truth Commission, though there are inconsistencies with risma in order to unite the country around its infant regard to how positively these mechanisms are viewed by institutions. The whole nation has its eyes turned to Timorese people. him right now. 10 SAGE Open Figure 9. East Timor primary legitimacy profile: “People” theme Figure 10. East Timor primary sources: “Australia” theme with Xanana Gusmao concept directly to the marginalization suffered by people when offi- 52. /east timor primary/human rights and post-conflict cially denied voice: transitional justice in east timor~6.html/1/1_947 10. global voices online (1)~1.html/1/1_89 Gusmão’s personal charisma and moral authority increases his influence beyond the formal powers of The catastrophic decision to make Portuguese the the law. national language of East Timor perfectly illustrates the dogmatism and unreality of Alkatiri’s approach. This is in contrast to the official map, where no concepts This decision disfranchised young East Timorese who relating to individual leaders appeared on the official map. speak Tetun, Indonesian or English. While the concept “leaders” was found on the academic map, quotes indicated this concept to be mostly about the Another theme that emerged was an East Timor theme leaders of UNTAET. in which an “Australian” concept was prominent (see Issues pertaining to voice also arose in the primary legiti- Figure 10). macy profile. A query of the concept “including” across the Closer inspection of the text associated with this theme primary texts yielded the following quote, again referring to revealed that the dominant perception was one of suspicion a general theme of there being a gap in understanding toward Australia’s involvement in East Timor. The text data between locals and official institutions: indicated that from the time of the launch of UNTAET, there was speculation that the Australian government was more 24. /east timor primary/Timor ba nafatin~3. interested in Timor’s natural gas and oil supplies than in a html/1/1_376 humanitarian mission. The strength of these reactions arose from the perception that Australia was the driving force There is a visible lack of communication between behind the formation of UNTAET, which effectively relieved most Western-type institutions, including those cre- Timorese people from the governance of their country. Due ated locally, and a considerable part of the East to the scope of UNTAET, there was evidence of specific con- Timorese society. cerns about Australia exerting control over East Timor: Another common idea relating to voice that emerged in 12. /east timor primary/AR2006060201400_pf~1. the East Timor primary literature was the choosing of html/1/1_40 Portuguese as the national language. This was perceived in East Timor as provocative and elitist, given that the majority Now the Australians have disembarked with Police, of Timorese spoke Tetun or Indonesian, with few speaking investigators, and magistrates. They will want to take Portuguese. The concepts of “Portuguese” and “English” over the Justice and then the Public Administration. were queried together to tap text revolving around language. Do not fool yourselves: there is a strategy behind that. The quote below was especially interesting as it alluded They did exactly the same in the Solomon Islands. Fisk et al. 11 With the pretext of fighting the gangs, they pushed the rights violations where they operate in an unregulated Police against the military and were able to put in the way without adequate protection. Government whoever they wanted to. Primary legitimacy profile As stated previously, a significant issue for locals was the concept of human rights and justice, as opposed to the aca- 34. /east timor primary/human rights and post-conflict demic figures where these themes were not as prominent. transitional justice in east timor~3.html/1/1_377 The lack of faith in the judicial system in East Timor was overwhelmingly reflected in the primary literature and was through traditional mechanisms of mediation, and this identified as a potential obstacle to institutional reform in practice continues today, partly because the formal East Timor. The following is a fraction of the opinions repre- justice system still does not work properly, and partly sented in the primary text regarding the judicial system in because people have no confidence in the formal jus- East Timor: tice system. There is also little understanding of the legal process even among educated people in Dili, let 30. /east timor primary/human rights in east timor~5. alone people in the villages. html/1/1_755 Tensions between traditional and formal systems of jus- Fear and mistrust of authority and lack of faith in the tice have been recognized more broadly with the peacebuild- system to deliver justice are consequences of the long ing literature (e.g., Baker & Scheye, 2007, 2009). The desire occupation. for a liberal democracy and functioning institutions while still maintaining a sense of traditional culture was an issue A related topic to perceptions of justice was the issue of alluded to in the primary sources and was raised as a signifi- using the reconstructed judicial system as opposed to relying cant challenge for reconstruction efforts. on traditional methods of justice. This issue was common Summary: Primary sources. Primary sources emphasized across academic, official, and primary sources. The feeling human rights and the judiciary as the most important ele- of foreign forces changing the way a central issue such as ments of obtaining legitimacy through peacebuilding. As justice is handled reflects social identity processes, that is, with the official sources, there were some conflicting opin- “our way” versus “your way.” At the same time, there exist ions about the use of traditional justice versus establishing a inherent tensions in dealing with lawbreakers via traditional, centralized justice system. Xanana Gusmao was singled out nonjudicial justice mechanisms, as they potentially under- for his influential position as a leader and as a source of mine the legitimacy of the “official” justice system and can enhancing government legitimacy via charismatic authority. create tensions between the operation of formal and informal Notions of voice were discussed, particularly with regard to justice processes. The following quotes are an example of local disenfranchisement over the selection of Portuguese as opinions relating to this tension, taken from each text source: an official language and were tied to social identity factors, as Portuguese is seen as an out-group language, spoken by Academic legitimacy profile: former colonizers and by the elite. This link between voice and social identity was particularly strong in our primary 4. /east timor academic/united nations transitional sources. Further connections between legitimacy and social administration in east timor~2.html/1/1_346 identity were found in discussions of UNTAET, where locals expressed disapproval of the intervention and whether for- Because the traditional resolution of gender-related eign interests dominated the United Nations intervention. crimes was so deeply etched into East Timorese legal culture, respondents considered police intervention in Discussion such matters to be disrespectful of traditional law and inconsistent with customary norms. In East Timorese Implications for Research and Policy culture only certain types of criminals go to the police. The development of legitimacy profiles across academic, Official legitimacy profile: official, and primary texts related to peacebuilding in East Timor were characterized by a disjunction between the ide- 13. /east timor official/www.laohamutuk.org/ als of the academic literature, the aims of reconstruction Bulletin/2001/Oct/bulletinv2n6~1.html/1/1_80 programs, and the reality on the ground. Each text source and the development of legitimacy profiles represent a dif- As Amnesty International noted in a recent report on ferent form of aggregation as it pertains to varying stake- East Timor, the use of alternative, non-judicial crimi- holder assessments of peacebuilding. Our concern was to nal justice mechanisms can lead to serious human explore how notions of domestic legitimacy were under- 12 SAGE Open stood and conceived within the statebuilding and peace- allowing for triangulation. It is important to note that we do building literature. A similar methodological approach could not see this analysis as a definitive guide to study local pro- be used to explore other facets of postconflict peacebuilding. cesses in Timor-Leste. Rather the methodology should be Our results indicated that the academic literature dis- seen as a way to explore predefined theories or concepts cussed legitimacy with equal attention across a broad range through rapid content analysis of a large literature. of institutions, including economic, health, and security. The theory of domestic legitimacy as employed in this The official literature, however, focused on government, project and the intersecting variables of social identity and and the primary literature focused on human rights and the voice provide a useful basis by which to analyze available judiciary. The presence of Timorese individuals such as text sources. The text analysis allowed us to understand Xanana Gusmao in primary profiles—but not academic or how these variables are linked and identity their key charac- official—suggests a possible underestimation of the power teristics. Such results give insight into how the short- and of the individual to influence perceptions of legitimacy, via long-term success of postconflict peacebuilding can be what can be termed charismatic authority. Though building assessed. Our methodology indicates how agencies (whether participation and increasing voice is discussed in the aca- international, governmental, or nongovernmental) can demic literature and explicitly stated as an objective in the undertake site assessments through the use of available tex- official literature, the primary literature shows that East tual sources prior to engaging in interventions in the field. Timorese still struggle with the impression of having little This can help identify local issues that may have a bearing influence over the rebuilding of key institutions. This leads on the way agencies are received and accepted by specific to perceptions of systemic bias and ineffectiveness, which populations, and which may also complicate the implemen- thereby undermines domestic legitimacy. There is a strong tation of policies and programs. For example, agencies can indication that a lack of trust regarding Australia’s involve- undertake similar country assessments when looking at ment in the reconstruction of East Timor impacted on the entering unstable regions and could use a similar methodol- legitimacy of UNTAET. ogy to assess not just country stability but also specific In the case of East Timor, the text analysis indicated that issues that may affect their operations in particular regions. suspicions abound of outsiders and their motives for involve- The United Nations and NGOs could perform similar ment, at the stage of the international intervention and subse- assessments as undertaken in this article to help tailor devel- quently in reconstruction. This was exacerbated when other opment programs to specific community needs. Researchers high-status groups were operating in the country in question. can also undertake similar analysis of the type outlined to For example, locals in East Timor were generally accepting prepare for in-country fieldwork or in the design of data col- of UN involvement, though it was evident that this support lection instruments. Rapidly reviewing texts through con- was tempted by perceptions that foreign governments were tent analysis can help researchers systematically grasp an “pulling the strings” of the operation. One can conclude that understanding of a certain research field, whose body of social identity processes are at work, arising from the innate literature maybe beyond the capacity of researchers to distrust of the motives of particular out-groups as repre- remain updated on. Using computer-aided content analysis sented by international agencies and their representatives in software to develop profiles on core concepts should be East Timor. understood as an ongoing process with profiles updated It needs to be recognized that there are limitations with when newly discovered and emerging text sources are the methodology adopted in this project. For example, accessed. This can help researchers or agencies to measure access to textual data was limited to English language change over time, thus allowing for a more longitudinal sources. Although many of the sources (especially blogs and assessment of postconflict peacebuilding. forums) included English translations of Portuguese and Tetun, sections of local opinions may not be represented in Conclusion this analysis. Similarly, populations that do not have access to technologies that allow them to communicate their opin- Our methodology highlights how content analysis tools (i.e., ions to a wider audience will not have been included. In Leximancer) can provide a means by which to assess impor- addition, secondary sources are the subject of filtering and tant features of postconflict peacebuilding. When combined editing processes of their authors and in the case of official with a specific theory and methodology by which to catego- literature reflect the bureaucratic and authorized stance of rize text sources, content analysis can allow one to undertake authorities. Hence, the accuracy of official reports and doc- concise and efficient evaluation of texts within a particular uments need to be kept in mind, given there can be variance research field. This is not to argue that assessments such as between what is officially stated and what actually occurs in the one undertaken here are superior to “on the ground” practice. We have attempted to correct for this potential bias fieldwork. It provides one methodology among many that by exploring key concepts across a variety of data sources, could be adopted as part of a mixed-methods approach. Fisk et al. 13 Australian Journal of International Affairs Appendix Brown University Thesis: Transitional Justice: The Sources List Case of East Timor Asian Law Journal East Timor Primar y Development and Change International Peacekeeping 1999 Horrors of East Timor Journal of Intervention and Statebuilding Voices of East Timor Radical Society Timor Ba Nafatin Journal of Peacebuilding and Development East Timor Eyewitnesses American Political Science Review In Asia Forum International Security East Asia Forum East Timor Law and Justice Bulletin East Timor Official East Timor Women Global Voices Online United Nations Security Council East Timor Today United Nations Transitional Authority in East Timor Alcofa Blogspot United Nations Development Program East Timor Directory The Alola Foundation Australasia News Commission for Reconstruction Timor The World Today International Centre for Transitional Justice Economist Asia Ministry of Social Solidarity Asia Times (stories interviewing locals) Statement to UN by Jose Ramos-Horta AM Archive (stories interviewing locals) Laohamutuk–Timor-Leste Institute for Reconstruction WorldFocus (stories interviewing locals) Monitoring and Analysis Rural News (stories interviewing locals) Ministry of Agriculture and Fisheries Science in a Suitcase (stories interviewing locals) Timor-Leste Institute for Development Monitoring Post Global (stories interviewing locals) and Analysis BBC East Timor (stories interviewing locals) Australian Centre for Peace and Conflict Studies PM (stories interviewing locals) East Timor and the United Nations (ETAN) BBC News Asia Pacific (stories interviewing locals) United Nations Mission of Support in East Timor: Foreign Correspondent (stories interviewing locals) UNMISET 730 Report (stories interviewing locals) UNOTIL: UN Office in Timor-Leste Timor Alliance for International Tribunal East Timor Academic International Federation for East Timor ReliefWeb: Timor-Leste Asia-Pacific Journal on Human Rights and the Law Asian Development Bank Journal of Contemporary Asia International Organization for Migration (IOM) International Law and Politics UNICEF–East Timor Democratization UN High Commission for Refugees Journal of Peace Research World Food Program Deutsche Gesellschaft für Technische Zusammenarbeit World Health Organization Oxford University Press World Bank Land Warfare Studies Centre The Finnish Institute of International Affairs (FIIA) Authors’ Note The London School of Hygiene and Tropical Medicine Foreign Service Journal The opinions express in this article are those of the authors. International Journal of Transitional Justice Declaration of Conflicting Interests Third World Quarterly Development in Practice The author(s) declared no potential conflicts of interest with Australian and New Zealand Journal of Psychiatry respect to the research, authorship, and/or publication of this International Conference Paper: Regional and global article. challenges of reconciliation Funding Australian Journal of Human Rights Symposium paper: State reconstruction after civil The author(s) disclosed receipt of the following financial support conflict for the research and/or authorship of this article: This project was 14 SAGE Open supported through a financial award provided by United States Air Harmer, A., & Frith, R. (2009). Walking together toward indepen- Force Office of Scientific Research (AFOSR) and the Unites States dence: A civil society perspective on the United Nations Admin- Asian Office of Aerospace Research and Development (AOARD)— istration in East Timor. Global Governance, 15, 239-258. Award Number FA2386-08-1-4094 and AOARD-124052. Hewstone, M., Rubin, M., & Willis, H. (2002). “Intergroup bias.” Annual Review of Psychology, 53, 575-604. Note Higashi, D. (2009). The challenge of constructing legitimacy in 1. Subsequently, an oil and gas reserve deal was struck with peacebuilding: Case of Timor-Leste (CIR Working Paper No. Australia, delivering the Timorese government around 4 bil- 48). Vancouver, British Columbia, Canada: Centre of Interna- lion dollars and forming the East Timor Petroleum Fund. tional Relations. Recently the Timorese government dipped into the petroleum Hornsey, M. J. (2008). Social identity theory and self-categorization fund to reimburse displaced Timorese being repatriated after theory: A historical review. Social and Personality Psychology the most recent conflict in 2006. Assessment of primary texts Compass, 1, 204-222. did suggest that this is fueling disenchantment among those Hornsey, M. J., & Imani, A. (2004). Criticizing groups from the who remained in their villages during that time. Perceptions inside and the outside: An identity perspective on the intergroup of distributive justice relating to the Petroleum Fund handouts sensitivity effect. Personality and Social Psychology Bulletin, will eventually, if not already, have the potential to affect the 30, 365-383. perceptions of governmental legitimacy. Krippendorff, K. (2004). Content analysis: An introduction to its methodology. Thousand Oaks, CA: SAGE. References Lemay-Hebert, N. (2009). Statebuilding without nation building? Baker, B., & Scheye, E. (2007). Multi-layered justice and security Legitimacy, state failure and the limits of the institutionalist delivery in post-conflict and fragile states. Conflict, Security & approach. Journal of Intervention and Statebuilding, 3, 21-45. Development, 7, 503-528. Leximancer Manual. (2010). Leximancer manual version 3.5. Baker, B., & Scheye, E. (2009). Access to justice in a post-con- Retrieved from https://leximancer.com/gallery/ flict state: Donor-supported multidimensional peacekeeping in Mersiades, M. (2005). Peacekeeping and legitimacy: Lessons from Southern Sudan. International Peacekeeping, 16, 171-185. Cambodia and Somalia. International Peacekeeping, 12, 205- Barnet, M., & Zurcher, C. (2009). The peacebuilding’s contract: 221. How external statebuilding reinforces weak statehood. In R. Olssen, L. (2009). Gender equality and UN peace operations in Paris & T. D. Sick (Eds.), The dilemmas of statebuilding: Con- Timor Leste. Leiden, Netherlands: KoninklijkeBrill NV. fronting the contradictions of postwar peace operation (pp. 23- Paris, R., & Sick T. D. (2009). The dilemmas of statebuilding: Con- 52). New York, NY: Routledge. fronting the contradictions of postwar peace operation (pp. 53- Brewer, M. (1979). In-group bias in the minimal intergroup situa- 78). New York, NY: Routledge. tion: A cognitive-motivational analysis. Psychological Bulletin, Richmond, O. P., & Franks, J. (2008). Liberal peacebuilding in 86, 307-324. Timor Leste: The emperor’s new clothes? International Peace- Brinkerhoff, D. W. (2005). Rebuilding governance in failed states keeping, 15, 185-200. and post-conflict societies: Core concepts and cross-cutting Roberts, D. (2008). Post-conflict statebuilding and state legitimacy: themes. Public Administration and Development, 25, 3-14. From negative to positive peace? Development and Change, 39, Chesterman, S. (2004). You, the people: The United Nations, tran- 537-555. sitional administration and statebuilding. Oxford, UK: Oxford Rubinstein, R. A. (2008). Peacekeeping under fire: Culture and University Press. intervention. Boulder, CO: Paradigm Publishers. Chesterman, S. (2007). Ownership in theory and in practice: Trans- Saldanha, J. M. (2008). Anatomy of political parties in Timor- fer of authority in UN statebuilding operations. Journal of Leste. In R. Rich, L. Hambly & M. G. Morgan (Eds.), Political Intervention and Statebuilding, 1, 3-26. parties in the Pacific Islands (pp. 69-81). Canberra, Australia: Chesterman, S., Ignatieff, M., & Thakur, R. (2005). Making states Pandanus Books. work: State failure and the crisis of governance. Tokyo, Japan: Smith, A. E. (2003). Automatic extraction of semantic networks United Nations University. from text using Leximancer. In Proceeding NAACL-Demon- Cohen, G. L. (2003). Party over policy: The dominating impact of strations ‘03 Proceedings of the 2003 Conference of the North group influence on political beliefs. Journal of Personality and American Chapter of the Association for Computational Lin- Social Psychology, 85, 808-822. guistics on Human Language Technology: Demonstrations Croissant, A., & Schacter, T. (2008). The nationalization of political (Vol. 4, pp. 23-24). Retrieved from http://dl.acm.org/citation. parties and party systems in Asian neo-democracies. Political cfm?id=1073439 Quarterly, 40, 641-668. Smith, A. E., & Humphreys, M. S. (2006). Evaluation of unsuper- Foddy, M., Platow, M. J., & Yamagishi, T. (2009). Group-based vised semantic mapping of natural language with Leximancer trust in strangers: The role of stereotypes and expectations. Psy- concept mapping. Behavior Research Methods, 38, 262-279. chological Science, 20, 419-422. Stockwell, P., Colomb, R. M., Smith, A. E., & Wiles, J. (2009). Use Folger, R. (1977). Distributive and procedural justice: Combined of an automatic content analysis tool: A technique for seeing impact of “voice” and improvement on experienced inequity. both local and global scope. International Journal of Human Journal of Personality and Social Psychology, 35, 108-199. Computer Studies, 67, 424-436. Fisk et al. 15 identity processes, ethnicity, perceptions of trust and procedural Tajfel, H., & Turner, J. C. (1979). An integrative theory of intergroup justice impact on post-conflict statebuilding. conflict. In W. Austin & S. Worchel (Eds.), The social psychology of intergroup relations (pp. 33-47). Monterey, CA: Brooks/Cole. Dr. Adrian Cherney is a Senior Lecturer in Criminology, in the Travaglia, J. F., Westbrook, M. T, & Braithwaite, J. (2009). Imple- School of Social Science at the University of Queensland. One mentation of a patient safety incident management system major focus of his work is on institutional legitimacy and coopera- as viewed by doctors, nurses and allied health professionals. tion with authorities e.g. police and government. Research funded Health: An Interdisciplinary Journal for the Social Study of by the US Airforce examines the issue of institutional legitimacy Health, Illness and Medicine, 13, 277-296. and cooperation with legal authorities in post-conflict Nepal and Turner, J. C. (1991). Social influence. Milton-Keynes, UK: Open Afghanistan. He was recently awarded an Australian Research University Press. Council Discovery project to look at community cooperation in Tyler, T. (2006). Psychological perspectives on legitimacy and counter-terrorism. legitimation. Annual Review Psychology, 57, 375-400. United Nations. (2008). United Nations peacekeeping operations: Matthew J. Hornsey is a Professor of Social Psychology at the Principles and guidelines. New York, NY: United Nations, University of Queensland. His research interests are in the areas of Department of Peacekeeping Operations and the Department group processes and intergroup relations, with particular interests of Field Support. in (a) how people respond to trust-sensitive messages such as criti- United Nations Integrated Mission in Timor-Leste. (2010). Fac- cisms, recommendations for change, and gestures of remorse; and ing the future: Periodic report on human rights developments (b) the dynamic and sometimes tense relationship between indi- in Timor-Leste. July 1, 2009-June 30, 2010. New York, NY: vidual and collective selves. United Nations. Weber, M. (1964). The theory of social and economic organization. Dr. Andrew Smith is the creator of Leximancer, a software appli- New York, NY: Free Press. cation for text analysis. He is currently Leximancer Chief Scientist, Weber, R. P. (1990). Basic content analysis (2nd ed.). Newbury Adjunct Researcher within the School of Information Technology Park, CA: SAGE. and Electrical Engineering at UQ, and Senior Research Officer Bios within the Institute for Social Science Research (ISSR) in the UQ Faculty of Social and Behavioural Sciences. His research focuses Kylie Fisk is a PHD candidate in the School of Social Science, at on developing new ways to visualise and quantify the temporal the University of Queensland. Her PHD research is focusing on dynamics of communication. institutional legitimacy in post-conflict Nepal and how social http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png SAGE Open SAGE

Using Computer-Aided Content Analysis to Map a Research Domain: A Case Study of Institutional Legitimacy in Postconflict East Timor

Fisk, Kylie; Cherney, Adrian; Hornsey, Matthew; Smith, Andrew
SAGE Open , Volume 2 (4): 1 – Nov 22, 2012
Download PDF
15 pages

Loading next page...
 
/lp/sage/using-computer-aided-content-analysis-to-map-a-research-domain-a-case-WOPT1p1oz8

References (42)

Publisher
SAGE
Copyright
Copyright © 2022 by SAGE Publications Inc, unless otherwise noted. Manuscript content on this site is licensed under Creative Commons Licenses.
ISSN
2158-2440
eISSN
2158-2440
DOI
10.1177/2158244012467788
Publisher site
See Article on Publisher Site

Abstract

Content analysis is a powerful tool for investigating attitudes expressed in naturally occurring language data. It is a useful tool to help researchers develop an understanding of a specific research field through identifying how particular issues or topics have been conceptualized or where fieldwork can be limited or prohibitive. This is especially true for research on postconflict reconstruction, where large-scale quantitative surveying or metareviews of the literature can be prohibitive. The present study provides a case study of how a particular content analysis software program—Leximancer—was used to map factors associated with institutional legitimacy in postconflict societies. The case of Timor-Leste is used as an example. We examine texts at three levels of discourse: at the academic, official, and primary levels. Results indicate differing perspectives on legitimacy at each level of discourse. This article offers a snapshot of a potential method for understanding how particular topics are conceptualized within a specific research field and can thus help in the development of evaluation methods or data collection instruments. Keywords legitimacy, content analysis, research methods, postconflict societies, social identity such as ethnographic fieldwork or qualitative interviewing Introduction does provide opportunities to gauge local insights, they can Although the legal and international legitimacy of peace- suffer from small sample sizes and hence lack external valid- building missions is regarded as one of the most important ity. assets of efforts to rebuild failed states (Barnet & Zurcher, The aim of this article is to test a potential alternative 2009; Brinkerhoff, 2005; Chesterman, Ignatieff, & Thakur, approach (i.e., content analysis) that could be used as a 2005; Lemay-Hebert, 2009; Mersiades, 2005; Roberts, “complement” to other forms of data collection or as a stand- 2008; United Nations, 2008), there is a lack of research alone method. Content analysis examines the conceptual examining factors that influence domestic legitimacy among structure of existing opinions expressed in text-based sources local populations. Domestic legitimacy refers to whether of information. We recognize that such an approach has its peacebuilding processes (here we include statebuilding as limitations, particularly in postconflict contexts where low well) are accepted and supported by local groups. It incorpo- literacy rates, limited access to the Internet, and an embry- rates beliefs and attitudes and has a behavioral outcome, in onic print media mean local opinions can be underpresented that it impacts on people’s willingness to cooperate with in different text-based data sources. However it can provide institutional authorities. Currently there is no defining meth- a viable starting point for understanding how particular odology for assessing the attitudes of local groups and stake- issues or concepts are understood or conceived, which then holders toward reconstructed institutions and the agents provides useful data to prepare fieldwork methods or under- responsible for rebuilding them. Evaluation methods may take closer analysis. include population-based studies of public attitudes in post- University of Queensland, St Lucia, Australia conflict societies, though achieving a reliable nationwide sample in a postwar country is logistically difficult. By the Corresponding Author: time results are collected and analyzed, the shifts in political Adrian Cherney, School of Social Science, University of Queensland, St circumstances may have rendered the surveyed public opin- Lucia 4072, Australia Email: a.cherney@uq.edu.au ion irrelevant. Likewise, although in-country data collection 2 SAGE Open To gain insight into the issue of the legitimacy of recon- Saldanha, 2008). For example, the 2007 elections produced structed institutions, not only are local attitudes measured a majority vote for the FRETILIN (Frente Revolucionária via analysis of online forums and blogs but also other groups do Timor Leste Independente) political party (29% of votes; with varied involvement in peacebuilding processes are Higashi, 2009), yet the government was eventually formed included in our analysis: that is, “officials” (including local by a coalition government lead by popular Prime Minister government, United Nations, and nongovernmental organi- Xanana Gusmao. FRETILIN denounced the coalition as zation [NGO] text sources) and “academics” (journal arti- unconstitutional and illegitimate, with the legitimacy of the cles published by experts in the field). These three sources elected government remaining a dividing issue for locals of information can provide differing perspectives on the (Higashi, 2009). Timor-Leste is a useful study for assessing legitimacy of state and international institutions involved in institutional legitimacy after peacebuilding and reconstruc- postconflict reconstruction and in the process, allow one to tion efforts due to the mixed outcomes of the reconstruction gain insight into the rationale for those opinions. Leximancer, process, with it struggling to maintain basic legitimacy in a text analytics program, is used to facilitate content analy- the eyes of the local population. sis, with it helping to visualize the meaning of content in texts by extracting concepts, and then graphically mapping Theoretical Framework relationships between concepts (the use of Leximancer for content analysis will be explained in the “Method” section). Effective content analysis relies on a theoretical framework This study is exploratory in nature and we recognize that through which to collate textual data and categorize the there are limitations in our approach. Our aim is to demon- emergent ideas. As stated above, the focal point of the strate a specific methodology and how it is relevant to stud- content analysis in this article is domestic legitimacy, that ies on postconflict reconstruction and outline a replicable is, the extent to which Timorese perceive their authorities, approach for achieving this. institutions, and social arrangements to be appropriate, Timor-Leste has been chosen as a case study for this proper, and just (Brinkerhoff, 2005; Tyler, 2006), and how method due to its history of violent conflict, institutional these perceptions compare with those expressed by officials damage, and subsequent reconstruction. Indonesian forces and academics. M. Weber (1964) links legitimacy to a invaded East Timor in 1975 and occupied the country for 25 population’s willingness to comply with a system of rule, years, and Indonesian rule throughout this time was charac- and identifies three sources of legitimate authority: ratio- terized as oppressive, with the army silencing dissent. nal-legal authority, which describes a belief in the legality Regardless, opposition to the Indonesians increased in the of the rule and the right of the leaders to issue commands; 1980s, and in 1999, under strong local and international traditional authority, which describes a belief in traditional pressure, the Indonesian government announced it would systems of rule based on history; and charismatic authority, hold a referendum on granting independence to East Timor. a devotion to an individual leader with whom the populace While the independence vote was overwhelmingly passed, feel a personal connection, legitimizing the authority Indonesian soldiers and prointegration militia embarked on of their decisions (M. Weber, 1964). The perception of a campaign of violence, resulting in the death of 1,400 legitimacy is essential for social stability—when it exists in Timorese, the destruction of basic infrastructure, and the the thinking of people and groups, it leads them to defer to displacement of approximately 300,000 people. One month authorities, institutions, and social arrangements as right later, an international mission called INTERFET (Inter- and proper (Tyler, 2006). Legitimacy promotes voluntary national Force for East Timor), led by Australian troops, was cooperation between people and institutional authorities deployed to stabilize the country. This administration was fol- and leads citizens to accept decisions and rules, and also lowed by the establishment of UNTAET (United Nations encompasses the consent and acceptance of local actors to Transitional Authority in East Timor), which oversaw international interventions. This has an important bearing all aspects of governance of Timor-Leste. Democratic elec- on local popular support for rebuilt institutional authorities tions were held in 2002 and UNTAET was disbanded. such as the police, judiciary, and government (Brinkerhoff, Although the security situation in Timor-Leste remains 2005; Chesterman, 2004, 2007; Paris & Sisk, 2009; United fragile (including unrest in 2006 that led to the deaths of Nations, 2008). nearly 40 people and an attempted assassination of President Studies of domestic legitimacy have provided varied Jose Ramos-Horta in 2008), a recent UN Human Rights insights into the micro-level factors that influence the success Report praised improvements, especially in reconstructed of peacebuilding and statebuilding programs “on the ground” security and justice institutions (United Nations Integrated (e.g., Harmer & Frith, 2009; Mersiades, 2005; Rubinstein, Mission in Timor-Leste [UNMIT], 2010). Nonetheless, 2008). One factor contributing to perceptions of legitimacy is research has drawn attention to the problems Timor-Leste the notion of voice. Voice refers to the opportunity for groups has faced in establishing the legitimacy of its transitional to comment on or inform an authority’s decision, allowing authorities as well as the new democratic government them to have some level of input into processes that affect that followed (Croissant & Schacter, 2008; Olssen, 2009; them (Folger, 1977). Voice provides for some level of local Fisk et al. 3 control and is premised on the concept of local accountability The variables of voice and social identity have been and participation in the reconstruction process. This is essen- adopted for theoretical and methodological reasons. First, tial to ensuring that reconstruction efforts are perceived as we wish to employ a flexible theoretical framework to guide being aimed at meeting local needs and avoids the problem of the analysis of the qualitative data but still allow these data them simply being perceived as a vehicle for local elites and to speak for themselves by giving space to new emerging international or corporate interests (Chesterman, 2007; themes. Second, given the data comprise a large collection of Harmer & Frith, 2009; Richmond & Franks, 2008). In short, text collected from the Internet, there was a substantial voice builds trust within and across social groups, and is one amount of material that was not relevant to our general of the biggest predictors of whether people perceive pro- research question. Our theoretical framework helped to filter cesses to be just and in their best interests (Folger, 1977). out this irrelevant material. However, it needs to be stressed Hence, we have selected voice as one variable through which that the concepts and themes discovered within the data are to study domestic legitimacy. emergent, revealed through the use of Leximancer, a text Past research also indicates other factors that have the analytics software program. Although this software and the potential to influence perceptions of legitimacy. For example, analytical method underpinning it (described in the next sec- Rubinstein (2008) draws on a cultural perspective and argues tion) has been demonstrated to be a reliable and valid method that any explanatory framework aimed at conceptualizing of content analysis (e.g., see Smith & Humphreys, 2006), the local-oriented legitimacy needs to take account of the social interpretation of the results Leximancer produces still stratifications among local populations that generate bound- requires a theoretical lens through which to comprehend the aries along cultural and ethnic lines. These stratifications various patterns revealed in our data sources. As new sources make the engagement and inclusion of various local actors of data are added to Leximancer and different techniques of complicated and contested but is essential to achieving a sta- exploring the data are adopted, the content of concepts and ble peace. Social identity theory (Tajfel & Turner, 1979; themes revealed in the data can shift. A theoretical frame- Turner, 1991; see Hornsey, 2008, for a recent review) pro- work is essential to identifying generalizable results and vides further insight, proposing that people are quick to cate- interpreting instances that do not fit with consistent trends gorize the world into in-groups and out-groups, based on emerging from the data. religion, ethnicity, age, culture, socioeconomic status, and political beliefs. Furthermore, these group allegiances pro- Method duce systematic biases in perception and behavior (e.g., Brewer, 1979; Cohen, 2003; Hewstone, Rubin, & Willis, Content Analysis 2002; Tajfel & Turner, 1979). One consequence of these biases is that group membership can act as a heuristic that Content analysis is a research method that uses a set of tells people who can be trusted and who cannot, independent procedures to draw valid inferences from texts (R. P. Weber, of any history of interpersonal exchange. It is assumed that 1990). Such inferences can relate to the attitudinal and in-group members can be trusted because they are motivated behavioral responses of organizations or people to particu- to protect the common good. Out-group members, however, lar issues (Krippendorff, 2004; R. P. Weber, 1990). Content are instinctively mistrusted because it is assumed that they analysis techniques determine the presence of words or are motivated to protect their own constituency at one’s own concepts in collections of textual documents and can be expense (Foddy, Platow, & Yamagishi, 2009; Hornsey & used to break large amounts of materials into manageable Imani, 2004). These biases could potentially have profound categories (Leximancer Manual, 2010; Stockwell, Colomb, effects on the extent to which processes central to statebuild- Smith, & Wiles, 2009; Krippendorff, 2004). It can be used ing are seen as legitimate. This is because citizens can draw to understand the conceptual structure of a set of documents assumptions about which groups are responsible for rebuild- and identify the most important occurring themes evident ing institutions (i.e., who is “pulling the strings”). For exam- within text-based data relating to a particular domain (e.g., ple, a local population may find themselves speculating about political, social, or economic). who is “pulling the strings” when it comes to developing a constitution or a parliament: Is it the local populace, the Leximancer United States, the United Nations, the European Union, spoil- ers allied to the government, big business, or a complicated Leximancer is a content analysis computer software that combination of all of these? The answer to these questions identifies core concepts within textual data (conceptual (which can also be based on fact, myth, intuition, or conspir- analysis) and clarifies the properties of these concepts and acy) would be a key determinant of whether people see rebuilt how they are related (relational analysis). Leximancer identi- institutions as valid and legitimate. Hence, the second contex- fies what concepts exist in a set of texts, allowing concepts to tual factor that provides a focus for this study is social be automatically coded. For example, Leximancer can take a identity. set of 1,000 documents, containing millions of words, and 4 SAGE Open produce a map containing the 50 most important concepts assessments and official committee reports. Primary within that text, along with their relationships with each sources encompassed any text representing the viewpoint other. The overall advantage of Leximancer is that it analyzes of locals. To retrieve this information, a variety of sources textual data in a more reliable fashion and produces visual were used: online discussion forums, social media, per- representations that aid interpretation (Smith, 2003; Smith & sonal blogs, local online media (including comments sec- Humphreys, 2006; Travaglia, Westbrook, & Braithwaite, tions), and transcripts from investigative journalism 2009). programs in which locals were interviewed. Many of these Leximancer concepts and themes. A concept in Leximancer sources included texts translated from Portuguese and is a set of words that travel together (co-occur) in the text. Tetun into English. The aim of gathering primary text was Leximancer identifies relationships by the frequency with to access the perceptions of citizens: their thoughts, beliefs, which words occur together in a block of text and the fre- and attitudes to the conflict itself and subsequent recon- quency with which they occur separately. This means that struction. Nonetheless, “local” data in this study does not concepts themselves are emergent, rather than selected by reflect the entirety of Timorese society, given its reliance the author. Concepts that co-occur often within the same on Internet access and literacy levels. two-sentence block attract one another strongly and are clus- tered in the visual map produced by Leximancer. Similar Analysis Process concepts tend to settle together in close proximity, which aids interpretation by the researcher. Leximancer “themes,” Analyzing all text sources in Leximancer created an initial however, are clusters of concepts, denoted by circles on the exploratory figure. After perusing the resulting visual maps (see figures provided). Concepts that are not close figure, concepts deemed to be irrelevant were removed. We enough to any existing theme will begin their own theme. excluded common function words (and, not, etc.) that The size of themes can be controlled by the software user comprise a standard set of excluded words in Leximancer. and set to the level of explanatory detail desired. Each theme We also removed general terms (such as area, total, section) is named after the most prominent concept in that group, that were used commonly in the text but that did not con- which is also indicated by the largest dot in the theme tribute meaning to the concept maps. In addition, we cluster. merged most singular and plural words; the words culture and cultures were eventually merged because they were closely associated. It was particularly necessary to delete Data Sources for the Current Study concepts in the primary figure, as the content was down- A variety of sources were used for analysis. A total of 1,653 loaded in its entirety as HTML and therefore many tags, documents were included in the analysis, distributed equally headlines, and extraneous material were also included as between each source. Postcollection, all literature was orga- concepts. The content of these concepts were carefully nized by source (see appendix for source lists). The purpose examined to determine with certainty that a concept was of this organization by source was to allow for subsequent the result of irrelevant HTML material and not part of per- comparisons from differing points of view. For this reason, tinent text. After deleting these concepts, the analysis was the three categories of textual sources were deemed aca- run again. demic, official, and primary, to reflect the differing view- From this point, it was possible to assemble a list of points of those observing, leading, and personally affected words that could be used as a starting point for Leximancer by statebuilding. This is not to say that these groups are analysis, termed seed words. This technique is known as mutually exclusive, nor entirely homogeneous: for example, “profiling,” and is a means of defining the scope of the con- UN officials may publish in academia, “local” sources may tent analysis to include only text data related to the notion be politicized elites, and government and UN sources may of legitimacy. Given their diverse origin within the aca- have differing viewpoints within the same text. But for the demic literature, the concepts of interest were rarely purposes of analysis, these categories represent three impor- referred to explicitly across the three key categories of text tant populations who have different levels of involvement in, sources. Hence, it was necessary to manually develop lists and views of, statebuilding. of associated terms relating to the concepts of legitimacy, Academic literature was retrieved primarily from refer- voice, and social identity that could then be entered into eed political and social science journals (published between Leximancer. For example, the notion of voice was identi- 2000 and 2010). Official literature comprised text produced fied as associated with ownership and participation, terms by those responsible for delivering peacebuilding programs common to the field of peacebuilding (see Chesterman, in Timor-Leste. This included UN official documentation, 2007; Harmer & Frith, 2009). The denial of voice was iden- including resolutions, mandates, and progress reports. It tified as related to the opposite of these terms: for example, also included material sourced from government agencies dispossession and exclusion. Discussions in the peace- and NGOs, from specific project outlines to country building literature concerning ethnic identity and class Fisk et al. 5 Figure 1. East Timor academic legitimacy profile: Main themes Figure 2. East Timor academic legitimacy profile: provided terms commonly associated with notions of social “Reconstruction” theme identity. These terms were also entered into Leximancer and used to construct concept figures. 3./east timor academic/ Results By the early months of 2000, a growing crisis of Academic Sources legitimacy had begun to develop, catalyzing what appeared to be a radical reorientation in UNTAET’s The academic legitimacy profile figure indicated that differ- approach to political authority and participation. On ent institutions requiring rebuilding after conflict were given May 30, UNTAET announced its intention to move well-balanced coverage in the academic literature when toward a period of “co-government” prior to the trans- discussing perceptions of legitimacy (see Figure 1). The fer of full authority to the East Timorese. theme of “government” emerged as central on the visual figure, surrounded by overlapping themes encompassing 6./east timor academic/ other institutions: “health,” “economic,” “security,” and “police.” The legitimacy crisis that resulted from UNTAET’s As the concept visibility in Leximancer was increased, early paralysis confirms the observation by several the central theme that grouped concepts together relating to analysts that the first weeks of a mission can be crucial legitimacy was “government.” Among the top-ranking con- to its ultimate success. cepts that were identified as occurring with government were those connected to issues of voice, for example, “participa- It is interesting to note that UNTAET is the main institu- tion” and “include,” indicating voice as an important issue tion identified in the co-occurrence of legitimacy and crisis. identified in the academic literature. Also appearing in the This runs counter to previous research suggesting that the East Timor legitimacy profile were characteristics or proper- United Nations benefits from a level of legitimacy beyond ties of legitimate institutions, in accordance with academic that of unilateral and even multilateral foreign agents, with theory and research: “functioning,” “appropriate,” “respon- UNTAET enjoying high levels of support among Timorese sible,” “accountable,” and “democratic.” Closely related was people (Higashi, 2009). Given that UNTAET is the most a “reconstruction” theme, in which legitimacy was a central ambitious transitional administration in the history of the concept (see Figure 2). United Nations, we anticipate polarized reactions, and these A query on the legitimacy concept revealed a negative criticisms leveled by academics will also be examined in the co-occurring concept: “crisis.” Text retrieved from original context of the results from the official and local sources. An text sources showed that in the perception of the academic investigation of the notion of “local” voice and participation community, the erosion of the legitimacy of UNTAET, par- resulted in a “knowledge pathway” between “local” and ticularly in its governance role, had created instability in “consultation.” A pathway is a Leximancer feature that East Timor in the period of 2000-2002. Below are some describes a relationship between two concepts. A start con- example quotes from the “legitimacy” and “crisis” query: cept is selected, followed by an end concept, and the links 6 SAGE Open Figure 3. Knowledge pathway between “local” and “consultation” concepts, academic sources between them are illustrated. The relationships between element of postconflict reconstruction and is regarded as tied these concepts are best thought of as correlations, with text to the legitimacy of agencies directing the reconstruction segments describing the relationship between each concept processes and the emerging institutions allocated responsi- in the pathway and the next (for further information, see bility for governance. Implicit in these academic conceptual- Leximancer Manual, 2010). Figure 3 charts the links between izations are notions of active participation in the peacebuilding each step as well as provides illustrative quotes. process, but a central question is whether such forms of We can observe from this result that in the academic lit- active voice have been achieved to the satisfaction of local erature, voice is explicitly linked to notions of legitimacy, populations. This is essential to whether local groups will particularly in the context of civil society, capacity build- voluntarily cooperate and support the activities of agencies ing, and good governance. Also in the top 10 concepts co- involved in peacebuilding missions and defer to their occurring with legitimacy are the concepts of “local” and authority. Social identity processes are alluded to in these “power.” academic sources, though not explicitly linked with percep- Summary of academic texts. From the academic legitimacy tions of foreign forces in Timor-Leste. These initial results profiles, the notion of “voice” is considered an essential will be further explored in the official and local texts. Fisk et al. 7 Figure 4. East Timor official legitimacy profile: Main themes Figure 5. East Timor official legitimacy profile: “System” theme Official Sources In the legitimacy profile of official sources, the representa- tion of institutions was less specific than in the academic Overall, results indicated that improving voice (i.e., par- literature (Figure 4). The main themes evident in this litera- ticipation) is identified as a key aim of reconstruction by the ture were “government” and “system,” which between them majority of government, United Nations, and NGO sources. thematically grouped the majority of concepts to appear on However, the above quote illustrates an important problem this figure. with statebuilding: the perception that government attempts The “system” theme comprised the majority of concepts to increase voice are aimed at appeasing international stake- and called for more in-depth analysis (see Figure 5). As in holders, rather than underpinned by a sincere and good- the academic figures, it illustrated a variety of concepts asso- willed attempt to actually encourage local participation. The ciated with legitimacy that are particularly applicable to official sources also contained assessments of attempts to notions of voice (e.g., “access,” “consultation,” enhance local participation through donor supported pro- “cooperation”). grams, for example, CEP—Community Empowerment and Building participation among locals was stated as an aim Local Governance Project—which is administered by the of most United Nations, NGO, and government policy docu- World Bank and the Asian Development Bank. One of the ments relating to statebuilding in East Timor. “Participation” aims of CEP was to empower Timorese through the creation was 1 of the top 10 concepts associated with “institutions” in of village and subdistrict councils to make funding the official literature, with a 16% likelihood that the 2 con- decisions supporting the local development of projects. cepts would co-occur in the same text segment. Also the top However, in its attempt to empower locals, CEP may have 10 concepts co-occurring with “institutions” were related had the opposite affect, as indicated through the following concepts of “involved” and “consultation”—reflecting dif- quote: ferent elements of voice. Figure 6 below illustrates the sec- tion of the map where these concepts were located. 7 ./east timor official/ Figure 6 reveals relationships between “voice” concepts in the official sources. Voice is referred to in the context of CEP’s attempt to bypass preexisting governance struc- institutions, co-occurring with concepts such as involved, tures with a new Council structure was received consultation, and participation. Also within this cluster is the ambiguously by communities and in many cases, gen- concept “responsibility,” wherein official sources discuss erated local conflict that inhibited the project. Instead, empowerment through involving local institutions. For the CEP model should have adopted the positive fea- example, the following quote from a local NGO illustrates tures of Timorese local authorities and challenged its this point: regressive features through continuous dialogue. 24./east timor official Another key issue to emerge related to rebuilding legiti- macy in the official literature was holding appropriate inves- We hope that this time the government will put more tigations into human rights violators as part of transitional efforts into making this consultation a true beginning justice processes (see Figure 7). for popular participation in Timor-Leste and not just Within the official literature there were a number of pro- use it to please international institutions. cess-related issues identified as important to institutional 8 SAGE Open Figure 6. Official map section containing concepts “institutions,” “participation,” “involved,” and “consultation” Figure 7. Official sources: “International” theme containing human rights–related concepts legitimacy, one being the necessity for commissions or trials and legal experts relating to the perceived legitimacy of into human rights violators to meet international standards Indonesia–Timor Leste Commission of Truth and Friendship and principles of fairness in the eyes of the general popula- (2005-2008), which investigated human rights abuses by tion. The official figures revealed some doubts among NGOs Indonesia during the occupation of Timor-Leste: Fisk et al. 9 15./east timor official/ While various governments on friendly terms with Jakarta applauded the long-stalled beginning of the court, human rights advocates have been far more critical. There are too many loopholes that could pre- vent the effectiveness of the trial, such as the skill of the judges, the law, as well as the independency of judges from elements of power. There is also some evidence that this perceived lack of legitimacy is related to Indonesian interests dominating the Commission: 19./east timor official/ The trials have failed fundamentally in fulfilling the “truth function,” a central part of the mandate of Figure 8. East Timor primary legitimacy profile: Main themes human rights and war crimes tribunals. Indeed, by clinging to a version of the violence in East Timor that Primary Sources is accepted nowhere outside of Indonesia (and rejected there by many), the trials have lost a unique opportu- The first figure resulting from the primary sources showed a nity to set the historical record straight, inform the contrast to the academic and official figures (Figure 8), with Indonesian public of the accountability of their institu- an emphasis on more humanitarian concerns: for example, tions for the gross human rights violations perpetrated “culture,” “rights,” “people,” “group,” “agree.” in their name, and provide a basis for reconciliation. Results from the primary texts indicated that for the Timorese people, a sense of transitional justice was of great This provides preliminary support for the social identity concern. Close inspection of the main themes in the primary hypothesis, with the motives and outcomes of the Commission literature showed that human rights considerations were cen- being innately distrusted and disputed by local Timorese due tral to perceptions of legitimacy and connected to almost all to suspicion over who is in control of the process. other issues in the profile. Summary: Official sources. Concepts relating to voice (i.e., When the theme size was decreased to reveal smaller participation and consultation) were closely bound to notions clustered themes, variations on Xanana Gusmao’s name of legitimacy in the official profiles. In some cases, govern- appeared on the primary figure within the people theme, very ment attempts to run community programs aimed at increas- close to a “leadership” concept (see Figure 9). Xanana ing voice were viewed as attempts to appease international Gusmao is a central figure in Timorese politics, the previous stakeholders, rather than a genuine attempt to use local opin- leader of the armed resistance turned first head of state. ion to inform policy implementation. In this case, the con- Interestingly, his name did not appear on either the academic cept of voice is tied to social identity processes, with locals or official legitimacy profiles. His importance to the distrusting institutions due to perceptions that they are influ- Timorese people may reflect a source of legitimacy that Max enced by outsiders and reflect foreign interests. This points Weber (1964) calls “charismatic authority,” wherein legiti- to one of the key challenges peacebuilding missions face, in macy for a political party or movement is gained through the that while efforts might be made to provide opportunities for power of an individual’s personality. The following quotes locals to have voice through processes of consultation, if emphasize the important legitimating role of Gusmao: those strategies do not translate into actual action that reflect the preferences of local populations—compared with donor 49./east timor primary/global voices online (4)~1. agencies—then the process and the outcomes will be viewed html/1/1_69 with suspicion and loose domestic legitimacy. Official sources also emphasize the path to legitimacy through tran- It’s fair also to hope that the national hero Xanana sitional justice and forgiveness mechanisms such as the Gusmão makes the best use of his well known cha- Truth Commission, though there are inconsistencies with risma in order to unite the country around its infant regard to how positively these mechanisms are viewed by institutions. The whole nation has its eyes turned to Timorese people. him right now. 10 SAGE Open Figure 9. East Timor primary legitimacy profile: “People” theme Figure 10. East Timor primary sources: “Australia” theme with Xanana Gusmao concept directly to the marginalization suffered by people when offi- 52. /east timor primary/human rights and post-conflict cially denied voice: transitional justice in east timor~6.html/1/1_947 10. global voices online (1)~1.html/1/1_89 Gusmão’s personal charisma and moral authority increases his influence beyond the formal powers of The catastrophic decision to make Portuguese the the law. national language of East Timor perfectly illustrates the dogmatism and unreality of Alkatiri’s approach. This is in contrast to the official map, where no concepts This decision disfranchised young East Timorese who relating to individual leaders appeared on the official map. speak Tetun, Indonesian or English. While the concept “leaders” was found on the academic map, quotes indicated this concept to be mostly about the Another theme that emerged was an East Timor theme leaders of UNTAET. in which an “Australian” concept was prominent (see Issues pertaining to voice also arose in the primary legiti- Figure 10). macy profile. A query of the concept “including” across the Closer inspection of the text associated with this theme primary texts yielded the following quote, again referring to revealed that the dominant perception was one of suspicion a general theme of there being a gap in understanding toward Australia’s involvement in East Timor. The text data between locals and official institutions: indicated that from the time of the launch of UNTAET, there was speculation that the Australian government was more 24. /east timor primary/Timor ba nafatin~3. interested in Timor’s natural gas and oil supplies than in a html/1/1_376 humanitarian mission. The strength of these reactions arose from the perception that Australia was the driving force There is a visible lack of communication between behind the formation of UNTAET, which effectively relieved most Western-type institutions, including those cre- Timorese people from the governance of their country. Due ated locally, and a considerable part of the East to the scope of UNTAET, there was evidence of specific con- Timorese society. cerns about Australia exerting control over East Timor: Another common idea relating to voice that emerged in 12. /east timor primary/AR2006060201400_pf~1. the East Timor primary literature was the choosing of html/1/1_40 Portuguese as the national language. This was perceived in East Timor as provocative and elitist, given that the majority Now the Australians have disembarked with Police, of Timorese spoke Tetun or Indonesian, with few speaking investigators, and magistrates. They will want to take Portuguese. The concepts of “Portuguese” and “English” over the Justice and then the Public Administration. were queried together to tap text revolving around language. Do not fool yourselves: there is a strategy behind that. The quote below was especially interesting as it alluded They did exactly the same in the Solomon Islands. Fisk et al. 11 With the pretext of fighting the gangs, they pushed the rights violations where they operate in an unregulated Police against the military and were able to put in the way without adequate protection. Government whoever they wanted to. Primary legitimacy profile As stated previously, a significant issue for locals was the concept of human rights and justice, as opposed to the aca- 34. /east timor primary/human rights and post-conflict demic figures where these themes were not as prominent. transitional justice in east timor~3.html/1/1_377 The lack of faith in the judicial system in East Timor was overwhelmingly reflected in the primary literature and was through traditional mechanisms of mediation, and this identified as a potential obstacle to institutional reform in practice continues today, partly because the formal East Timor. The following is a fraction of the opinions repre- justice system still does not work properly, and partly sented in the primary text regarding the judicial system in because people have no confidence in the formal jus- East Timor: tice system. There is also little understanding of the legal process even among educated people in Dili, let 30. /east timor primary/human rights in east timor~5. alone people in the villages. html/1/1_755 Tensions between traditional and formal systems of jus- Fear and mistrust of authority and lack of faith in the tice have been recognized more broadly with the peacebuild- system to deliver justice are consequences of the long ing literature (e.g., Baker & Scheye, 2007, 2009). The desire occupation. for a liberal democracy and functioning institutions while still maintaining a sense of traditional culture was an issue A related topic to perceptions of justice was the issue of alluded to in the primary sources and was raised as a signifi- using the reconstructed judicial system as opposed to relying cant challenge for reconstruction efforts. on traditional methods of justice. This issue was common Summary: Primary sources. Primary sources emphasized across academic, official, and primary sources. The feeling human rights and the judiciary as the most important ele- of foreign forces changing the way a central issue such as ments of obtaining legitimacy through peacebuilding. As justice is handled reflects social identity processes, that is, with the official sources, there were some conflicting opin- “our way” versus “your way.” At the same time, there exist ions about the use of traditional justice versus establishing a inherent tensions in dealing with lawbreakers via traditional, centralized justice system. Xanana Gusmao was singled out nonjudicial justice mechanisms, as they potentially under- for his influential position as a leader and as a source of mine the legitimacy of the “official” justice system and can enhancing government legitimacy via charismatic authority. create tensions between the operation of formal and informal Notions of voice were discussed, particularly with regard to justice processes. The following quotes are an example of local disenfranchisement over the selection of Portuguese as opinions relating to this tension, taken from each text source: an official language and were tied to social identity factors, as Portuguese is seen as an out-group language, spoken by Academic legitimacy profile: former colonizers and by the elite. This link between voice and social identity was particularly strong in our primary 4. /east timor academic/united nations transitional sources. Further connections between legitimacy and social administration in east timor~2.html/1/1_346 identity were found in discussions of UNTAET, where locals expressed disapproval of the intervention and whether for- Because the traditional resolution of gender-related eign interests dominated the United Nations intervention. crimes was so deeply etched into East Timorese legal culture, respondents considered police intervention in Discussion such matters to be disrespectful of traditional law and inconsistent with customary norms. In East Timorese Implications for Research and Policy culture only certain types of criminals go to the police. The development of legitimacy profiles across academic, Official legitimacy profile: official, and primary texts related to peacebuilding in East Timor were characterized by a disjunction between the ide- 13. /east timor official/www.laohamutuk.org/ als of the academic literature, the aims of reconstruction Bulletin/2001/Oct/bulletinv2n6~1.html/1/1_80 programs, and the reality on the ground. Each text source and the development of legitimacy profiles represent a dif- As Amnesty International noted in a recent report on ferent form of aggregation as it pertains to varying stake- East Timor, the use of alternative, non-judicial crimi- holder assessments of peacebuilding. Our concern was to nal justice mechanisms can lead to serious human explore how notions of domestic legitimacy were under- 12 SAGE Open stood and conceived within the statebuilding and peace- allowing for triangulation. It is important to note that we do building literature. A similar methodological approach could not see this analysis as a definitive guide to study local pro- be used to explore other facets of postconflict peacebuilding. cesses in Timor-Leste. Rather the methodology should be Our results indicated that the academic literature dis- seen as a way to explore predefined theories or concepts cussed legitimacy with equal attention across a broad range through rapid content analysis of a large literature. of institutions, including economic, health, and security. The theory of domestic legitimacy as employed in this The official literature, however, focused on government, project and the intersecting variables of social identity and and the primary literature focused on human rights and the voice provide a useful basis by which to analyze available judiciary. The presence of Timorese individuals such as text sources. The text analysis allowed us to understand Xanana Gusmao in primary profiles—but not academic or how these variables are linked and identity their key charac- official—suggests a possible underestimation of the power teristics. Such results give insight into how the short- and of the individual to influence perceptions of legitimacy, via long-term success of postconflict peacebuilding can be what can be termed charismatic authority. Though building assessed. Our methodology indicates how agencies (whether participation and increasing voice is discussed in the aca- international, governmental, or nongovernmental) can demic literature and explicitly stated as an objective in the undertake site assessments through the use of available tex- official literature, the primary literature shows that East tual sources prior to engaging in interventions in the field. Timorese still struggle with the impression of having little This can help identify local issues that may have a bearing influence over the rebuilding of key institutions. This leads on the way agencies are received and accepted by specific to perceptions of systemic bias and ineffectiveness, which populations, and which may also complicate the implemen- thereby undermines domestic legitimacy. There is a strong tation of policies and programs. For example, agencies can indication that a lack of trust regarding Australia’s involve- undertake similar country assessments when looking at ment in the reconstruction of East Timor impacted on the entering unstable regions and could use a similar methodol- legitimacy of UNTAET. ogy to assess not just country stability but also specific In the case of East Timor, the text analysis indicated that issues that may affect their operations in particular regions. suspicions abound of outsiders and their motives for involve- The United Nations and NGOs could perform similar ment, at the stage of the international intervention and subse- assessments as undertaken in this article to help tailor devel- quently in reconstruction. This was exacerbated when other opment programs to specific community needs. Researchers high-status groups were operating in the country in question. can also undertake similar analysis of the type outlined to For example, locals in East Timor were generally accepting prepare for in-country fieldwork or in the design of data col- of UN involvement, though it was evident that this support lection instruments. Rapidly reviewing texts through con- was tempted by perceptions that foreign governments were tent analysis can help researchers systematically grasp an “pulling the strings” of the operation. One can conclude that understanding of a certain research field, whose body of social identity processes are at work, arising from the innate literature maybe beyond the capacity of researchers to distrust of the motives of particular out-groups as repre- remain updated on. Using computer-aided content analysis sented by international agencies and their representatives in software to develop profiles on core concepts should be East Timor. understood as an ongoing process with profiles updated It needs to be recognized that there are limitations with when newly discovered and emerging text sources are the methodology adopted in this project. For example, accessed. This can help researchers or agencies to measure access to textual data was limited to English language change over time, thus allowing for a more longitudinal sources. Although many of the sources (especially blogs and assessment of postconflict peacebuilding. forums) included English translations of Portuguese and Tetun, sections of local opinions may not be represented in Conclusion this analysis. Similarly, populations that do not have access to technologies that allow them to communicate their opin- Our methodology highlights how content analysis tools (i.e., ions to a wider audience will not have been included. In Leximancer) can provide a means by which to assess impor- addition, secondary sources are the subject of filtering and tant features of postconflict peacebuilding. When combined editing processes of their authors and in the case of official with a specific theory and methodology by which to catego- literature reflect the bureaucratic and authorized stance of rize text sources, content analysis can allow one to undertake authorities. Hence, the accuracy of official reports and doc- concise and efficient evaluation of texts within a particular uments need to be kept in mind, given there can be variance research field. This is not to argue that assessments such as between what is officially stated and what actually occurs in the one undertaken here are superior to “on the ground” practice. We have attempted to correct for this potential bias fieldwork. It provides one methodology among many that by exploring key concepts across a variety of data sources, could be adopted as part of a mixed-methods approach. Fisk et al. 13 Australian Journal of International Affairs Appendix Brown University Thesis: Transitional Justice: The Sources List Case of East Timor Asian Law Journal East Timor Primar y Development and Change International Peacekeeping 1999 Horrors of East Timor Journal of Intervention and Statebuilding Voices of East Timor Radical Society Timor Ba Nafatin Journal of Peacebuilding and Development East Timor Eyewitnesses American Political Science Review In Asia Forum International Security East Asia Forum East Timor Law and Justice Bulletin East Timor Official East Timor Women Global Voices Online United Nations Security Council East Timor Today United Nations Transitional Authority in East Timor Alcofa Blogspot United Nations Development Program East Timor Directory The Alola Foundation Australasia News Commission for Reconstruction Timor The World Today International Centre for Transitional Justice Economist Asia Ministry of Social Solidarity Asia Times (stories interviewing locals) Statement to UN by Jose Ramos-Horta AM Archive (stories interviewing locals) Laohamutuk–Timor-Leste Institute for Reconstruction WorldFocus (stories interviewing locals) Monitoring and Analysis Rural News (stories interviewing locals) Ministry of Agriculture and Fisheries Science in a Suitcase (stories interviewing locals) Timor-Leste Institute for Development Monitoring Post Global (stories interviewing locals) and Analysis BBC East Timor (stories interviewing locals) Australian Centre for Peace and Conflict Studies PM (stories interviewing locals) East Timor and the United Nations (ETAN) BBC News Asia Pacific (stories interviewing locals) United Nations Mission of Support in East Timor: Foreign Correspondent (stories interviewing locals) UNMISET 730 Report (stories interviewing locals) UNOTIL: UN Office in Timor-Leste Timor Alliance for International Tribunal East Timor Academic International Federation for East Timor ReliefWeb: Timor-Leste Asia-Pacific Journal on Human Rights and the Law Asian Development Bank Journal of Contemporary Asia International Organization for Migration (IOM) International Law and Politics UNICEF–East Timor Democratization UN High Commission for Refugees Journal of Peace Research World Food Program Deutsche Gesellschaft für Technische Zusammenarbeit World Health Organization Oxford University Press World Bank Land Warfare Studies Centre The Finnish Institute of International Affairs (FIIA) Authors’ Note The London School of Hygiene and Tropical Medicine Foreign Service Journal The opinions express in this article are those of the authors. International Journal of Transitional Justice Declaration of Conflicting Interests Third World Quarterly Development in Practice The author(s) declared no potential conflicts of interest with Australian and New Zealand Journal of Psychiatry respect to the research, authorship, and/or publication of this International Conference Paper: Regional and global article. challenges of reconciliation Funding Australian Journal of Human Rights Symposium paper: State reconstruction after civil The author(s) disclosed receipt of the following financial support conflict for the research and/or authorship of this article: This project was 14 SAGE Open supported through a financial award provided by United States Air Harmer, A., & Frith, R. (2009). Walking together toward indepen- Force Office of Scientific Research (AFOSR) and the Unites States dence: A civil society perspective on the United Nations Admin- Asian Office of Aerospace Research and Development (AOARD)— istration in East Timor. Global Governance, 15, 239-258. Award Number FA2386-08-1-4094 and AOARD-124052. Hewstone, M., Rubin, M., & Willis, H. (2002). “Intergroup bias.” Annual Review of Psychology, 53, 575-604. Note Higashi, D. (2009). The challenge of constructing legitimacy in 1. Subsequently, an oil and gas reserve deal was struck with peacebuilding: Case of Timor-Leste (CIR Working Paper No. Australia, delivering the Timorese government around 4 bil- 48). Vancouver, British Columbia, Canada: Centre of Interna- lion dollars and forming the East Timor Petroleum Fund. tional Relations. Recently the Timorese government dipped into the petroleum Hornsey, M. J. (2008). Social identity theory and self-categorization fund to reimburse displaced Timorese being repatriated after theory: A historical review. Social and Personality Psychology the most recent conflict in 2006. Assessment of primary texts Compass, 1, 204-222. did suggest that this is fueling disenchantment among those Hornsey, M. J., & Imani, A. (2004). Criticizing groups from the who remained in their villages during that time. Perceptions inside and the outside: An identity perspective on the intergroup of distributive justice relating to the Petroleum Fund handouts sensitivity effect. Personality and Social Psychology Bulletin, will eventually, if not already, have the potential to affect the 30, 365-383. perceptions of governmental legitimacy. Krippendorff, K. (2004). Content analysis: An introduction to its methodology. Thousand Oaks, CA: SAGE. References Lemay-Hebert, N. (2009). Statebuilding without nation building? Baker, B., & Scheye, E. (2007). Multi-layered justice and security Legitimacy, state failure and the limits of the institutionalist delivery in post-conflict and fragile states. Conflict, Security & approach. Journal of Intervention and Statebuilding, 3, 21-45. Development, 7, 503-528. Leximancer Manual. (2010). Leximancer manual version 3.5. Baker, B., & Scheye, E. (2009). Access to justice in a post-con- Retrieved from https://leximancer.com/gallery/ flict state: Donor-supported multidimensional peacekeeping in Mersiades, M. (2005). Peacekeeping and legitimacy: Lessons from Southern Sudan. International Peacekeeping, 16, 171-185. Cambodia and Somalia. International Peacekeeping, 12, 205- Barnet, M., & Zurcher, C. (2009). The peacebuilding’s contract: 221. How external statebuilding reinforces weak statehood. In R. Olssen, L. (2009). Gender equality and UN peace operations in Paris & T. D. Sick (Eds.), The dilemmas of statebuilding: Con- Timor Leste. Leiden, Netherlands: KoninklijkeBrill NV. fronting the contradictions of postwar peace operation (pp. 23- Paris, R., & Sick T. D. (2009). The dilemmas of statebuilding: Con- 52). New York, NY: Routledge. fronting the contradictions of postwar peace operation (pp. 53- Brewer, M. (1979). In-group bias in the minimal intergroup situa- 78). New York, NY: Routledge. tion: A cognitive-motivational analysis. Psychological Bulletin, Richmond, O. P., & Franks, J. (2008). Liberal peacebuilding in 86, 307-324. Timor Leste: The emperor’s new clothes? International Peace- Brinkerhoff, D. W. (2005). Rebuilding governance in failed states keeping, 15, 185-200. and post-conflict societies: Core concepts and cross-cutting Roberts, D. (2008). Post-conflict statebuilding and state legitimacy: themes. Public Administration and Development, 25, 3-14. From negative to positive peace? Development and Change, 39, Chesterman, S. (2004). You, the people: The United Nations, tran- 537-555. sitional administration and statebuilding. Oxford, UK: Oxford Rubinstein, R. A. (2008). Peacekeeping under fire: Culture and University Press. intervention. Boulder, CO: Paradigm Publishers. Chesterman, S. (2007). Ownership in theory and in practice: Trans- Saldanha, J. M. (2008). Anatomy of political parties in Timor- fer of authority in UN statebuilding operations. Journal of Leste. In R. Rich, L. Hambly & M. G. Morgan (Eds.), Political Intervention and Statebuilding, 1, 3-26. parties in the Pacific Islands (pp. 69-81). Canberra, Australia: Chesterman, S., Ignatieff, M., & Thakur, R. (2005). Making states Pandanus Books. work: State failure and the crisis of governance. Tokyo, Japan: Smith, A. E. (2003). Automatic extraction of semantic networks United Nations University. from text using Leximancer. In Proceeding NAACL-Demon- Cohen, G. L. (2003). Party over policy: The dominating impact of strations ‘03 Proceedings of the 2003 Conference of the North group influence on political beliefs. Journal of Personality and American Chapter of the Association for Computational Lin- Social Psychology, 85, 808-822. guistics on Human Language Technology: Demonstrations Croissant, A., & Schacter, T. (2008). The nationalization of political (Vol. 4, pp. 23-24). Retrieved from http://dl.acm.org/citation. parties and party systems in Asian neo-democracies. Political cfm?id=1073439 Quarterly, 40, 641-668. Smith, A. E., & Humphreys, M. S. (2006). Evaluation of unsuper- Foddy, M., Platow, M. J., & Yamagishi, T. (2009). Group-based vised semantic mapping of natural language with Leximancer trust in strangers: The role of stereotypes and expectations. Psy- concept mapping. Behavior Research Methods, 38, 262-279. chological Science, 20, 419-422. Stockwell, P., Colomb, R. M., Smith, A. E., & Wiles, J. (2009). Use Folger, R. (1977). Distributive and procedural justice: Combined of an automatic content analysis tool: A technique for seeing impact of “voice” and improvement on experienced inequity. both local and global scope. International Journal of Human Journal of Personality and Social Psychology, 35, 108-199. Computer Studies, 67, 424-436. Fisk et al. 15 identity processes, ethnicity, perceptions of trust and procedural Tajfel, H., & Turner, J. C. (1979). An integrative theory of intergroup justice impact on post-conflict statebuilding. conflict. In W. Austin & S. Worchel (Eds.), The social psychology of intergroup relations (pp. 33-47). Monterey, CA: Brooks/Cole. Dr. Adrian Cherney is a Senior Lecturer in Criminology, in the Travaglia, J. F., Westbrook, M. T, & Braithwaite, J. (2009). Imple- School of Social Science at the University of Queensland. One mentation of a patient safety incident management system major focus of his work is on institutional legitimacy and coopera- as viewed by doctors, nurses and allied health professionals. tion with authorities e.g. police and government. Research funded Health: An Interdisciplinary Journal for the Social Study of by the US Airforce examines the issue of institutional legitimacy Health, Illness and Medicine, 13, 277-296. and cooperation with legal authorities in post-conflict Nepal and Turner, J. C. (1991). Social influence. Milton-Keynes, UK: Open Afghanistan. He was recently awarded an Australian Research University Press. Council Discovery project to look at community cooperation in Tyler, T. (2006). Psychological perspectives on legitimacy and counter-terrorism. legitimation. Annual Review Psychology, 57, 375-400. United Nations. (2008). United Nations peacekeeping operations: Matthew J. Hornsey is a Professor of Social Psychology at the Principles and guidelines. New York, NY: United Nations, University of Queensland. His research interests are in the areas of Department of Peacekeeping Operations and the Department group processes and intergroup relations, with particular interests of Field Support. in (a) how people respond to trust-sensitive messages such as criti- United Nations Integrated Mission in Timor-Leste. (2010). Fac- cisms, recommendations for change, and gestures of remorse; and ing the future: Periodic report on human rights developments (b) the dynamic and sometimes tense relationship between indi- in Timor-Leste. July 1, 2009-June 30, 2010. New York, NY: vidual and collective selves. United Nations. Weber, M. (1964). The theory of social and economic organization. Dr. Andrew Smith is the creator of Leximancer, a software appli- New York, NY: Free Press. cation for text analysis. He is currently Leximancer Chief Scientist, Weber, R. P. (1990). Basic content analysis (2nd ed.). Newbury Adjunct Researcher within the School of Information Technology Park, CA: SAGE. and Electrical Engineering at UQ, and Senior Research Officer Bios within the Institute for Social Science Research (ISSR) in the UQ Faculty of Social and Behavioural Sciences. His research focuses Kylie Fisk is a PHD candidate in the School of Social Science, at on developing new ways to visualise and quantify the temporal the University of Queensland. Her PHD research is focusing on dynamics of communication. institutional legitimacy in post-conflict Nepal and how social

Journal

SAGE Open – SAGE

Published: Nov 22, 2012

Keywords: legitimacy; content analysis; research methods; postconflict societies; social identity

There are no references for this article.