Cross-cultural/national research is essential for both scholars and practitioners. Although the methodological issues specific to this research have long been acknowledged in the literature, recent studies confirm that the standards demanded by earlier studies have not been met. Accordingly, we review some of the most relevant methodological issues involved in cross-cultural/national research and provide guidelines for addressing these issues, offering a fresh perspective based on the new trends and suggestions from recent literature. This study seeks to encourage greater methodological rigor in survey research conducted across cultures and/or countries to develop more theoretically robust and managerially relevant international market research.
Avoid common mistakes on your manuscript.
The globalization of markets and marketing activities has increased the interest among both scholars and practitioners in conducting research across different cultures and/or countries. 1 The increasing internationalization of companies demands a high volume of accurate international information to assist decision making. This has given rise to the global market research industry having an estimated turnover of US $31.2 billion in 2010. 2
Undertaking research across nations and cultures presents several unique challenges. Important substantive questions for researchers and managers include the following: Are the constructs or theories investigated relevant in each research context? How should the instrument used to collect the data (for example, a questionnaire) be translated? What samples should be selected? How should the data be collected? Are the measures used in the study invariant in each unit of analysis investigated? The answers to these and other similar questions share a common denominator: the comparability or equivalence.
The main aim when conducting research across different cultures and/or countries is to establish comparability or equivalence at each stage of the research process. 3 A failure to establish this may bias results. Indeed, the interpretations of findings may be inconclusive, meaningless or misguiding. For instance, if an analysis of the wedding market does not take into account that social rituals such as weddings may vary from one country or culture to another, conclusions will be erroneous. To ensure equivalence, the methodological issues associated with this type of research have long been acknowledged in the literature. 4 However, past reviews of cross-cultural and cross-national empirical investigations in different research streams repeatedly highlight deficiencies. 5, 6, 7, 8, 9, 10 Specifically, the lack of methodological rigor at the different levels of the research process is the main problem found in the reviews. Contrary to what might be expected, recent studies confirm that the standards called for in past studies, such as achieving construct equivalence or assessing the measurement invariance of data collected, have not been met yet. 11, 12
There are several reasons for this lack of improvement in the methodology of this type of research: the lack of knowledge between researchers, the difficulty of setting and achieving equivalence at each stage of the research process, the methodological complexities involved in some procedures and the lack of clarity in the literature. 11, 13 More importantly, the significant number of sources about cross-cultural/national findings make it difficult and time-consuming to get a basic overview. 14
To overcome this and encourage greater methodological rigor in survey research conducted across cultures and/or countries, this article provides a review of the relevant literature. The objectives of this article are two-fold. First, to review some of the most relevant methodological issues involved in the main stages of cross-cultural/national marketing research. Second, to provide guidelines regarding these issues offering a fresh perspective based on the new trends and suggestions from recent literature.
This study focuses its review and recommendations in international survey research as one of the most prominent methods of data collection. 15 Following the traditions of others, 16, 17 the term cross-cultural research is used as a generic for all comparative studies that involve either different cultural groups or countries.
The article opens with a short overview of the equivalence concept. Then, as shown in Figure 1, the main methodological issues specific to cross-cultural research are organized around the main stages of the marketing research process. 18
The equivalence or comparability of data collected in different cultures and countries is critical in cross-cultural research. Data equivalence or comparability refers to ‘data that have, as far as possible, the same meaning or interpretation, and the same level of accuracy, precision of measurement, validity and reliability in all countries and cultures’. 19 Two approaches to equivalence are generally identified in the literature. 13 The first refers to those forms of equivalence that have to be addressed before and during data collection. The second approach focuses on the analysis of the data and examines the measurement equivalence or invariance of data already collected. Throughout the following sections, the main methodological aspects applicable within both approaches are reviewed. Important issues involved in each of the first five stages of the marketing research process described in Figure 1 are discussed. Suggestions and recommendations within each stage are also presented.
Two relevant methodological issues at this stage are the selection of the unit of analysis and the relevance of the research topic. Most research on international markets involves comparisons. Therefore, defining the unit of analysis, that is, selecting the relevant contexts to be compared is a priority in cross-cultural research. 1 Craig and Douglas 19 propose three aspects that need to be considered in defining the unit: the geographic scope of the unit (for example, country, region, and so on); the criteria for membership in the unit (for example, demographic or socio-economic characteristics, and so on); and the situational context (for example, specific socio-cultural settings, climate context, and so on). This section will focus on geographic scope, which needs to be chosen based on the purpose of the research.
Within the different geographical levels, the country level provides a practical and convenient unit for data collection. Thus, researchers mostly use this unit of analysis in their studies. However, the use of countries is criticized for several reasons. 1 First, countries are not always that relevant. Cities, regions or even the world may be more appropriate. Second, countries are not isolated or independent units. They develop and adopt similar practices and behaviors through numerous ways. Finally, the differences between countries in terms of economic, social or cultural factors, and the heterogeneity within countries can have unintended consequences.
The relevance of the topic in the selected units of analysis is more difficult and important than in domestic research, due to the unfamiliarity with the countries/cultures where the research is being conducted. The research topic should be equally important and appropriate in each context, and conceptually equivalent, an issue that will be addressed in the next section. 17, 20 Similarly, the relevance of constructs should be carefully evaluated. 1 This issue will help to avoid pseudoetic bias (that is, to assume that a measure developed in a context is appropriate in all the contexts).
Given the limitations of the use of the country, the consideration of different geographical units is suggested in the literature. As a result of advances in information and communication technology, improvements in physical communication and transportation, and the convergence of consumer needs, ‘national culture’ is less meaningful. 21, 22 Therefore, several authors call for the study of units of analysis, such as regions, communities or specific population segments (for example, teenagers), as well as the combination of multiple levels of units. 1, 23 However, these alternative units of analysis should not totally replace the use of national borders. Engelen and Brettel 24 justify their use based on existing theoretical and empirical evidence plus their managerial relevance, since organizations typically carry out their international activities along national borders.
If countries are used as unit of analysis, they should be ‘purposively selected to be comparable’, 1 taking into account those factors that may be relevant or affect the phenomenon being studied. Furthermore, researchers should beware of the degree of cultural interpenetration, that is, the extent to which the members of one country are exposed to another through different channels, such as the direct experience, the media or the experiences of others. It is also important to take into account the intra-national diversity to truly understand the phenomenon under investigation. Finally, the selection of the unit should be based on the objectives of the study rather than on convenience. 17, 25
Regarding the topic being investigated, Douglas and Craig 1 suggest removing the influence of the dominant culture. Researchers should isolate the tendency to allow their own beliefs and values to influence the question analyzed. It would help them to distinguish the relevant topics, constructs or relationships to be studied in each context. It is also important to identify the role of mediating and moderator factors embedded in each socio-cultural context and assess how this can be related to the focal topic. For instance, a study exploring the purchase intention of foreign products should consider to what extent the image of the country of origin affects this intention.
Two main alternative approaches have dominated cross-cultural research in social sciences: emic and etic. 26, 27 The emic approach examines the phenomenon studied from within a specific context (for example, culture, country). It holds that theory, constructs, and so on, are specific to this context. Hence, this approach requires developing specific measures for each unit of analysis and, taken to its extreme, limited or no comparisons are possible. For instance, this approach would imply developing different measures of a construct, such as brand personality, for each context. By contrast, the etic approach examines the phenomenon analyzed from outside a specific context (for example, culture, country). It is concerned with assessing universal constructs, theories, and so on, and allows for developing universal measures that can be applied to all contexts. That is, following with the previous example, this approach would imply using the same instrument to measure a construct, such as brand personality, regardless of the context. Although the emic approach offers more reliability and internal validity, the etic approach is considered more practical, in terms of time and cost. In addition, it makes comparisons easier and increases external validity. 3 Thus, researchers often use the etic approach. Accordingly, theories, conceptual models and research designs used in one culture or country are applied in others in the same way.
Researchers suggest using a combined emic-etic or a derived etic approach. 28, 29 In this approach, once emic dimensions are generated, they are analyzed to determine whether comparisons can be appropriately made. Similarly, alternative iterative approaches have been proposed in the literature, providing comparability without ignoring emic factors. Douglas and Craig 1 propose two approaches. The ‘linked emic model’ takes multiple local contexts as its base point and seeks to incorporate culture-specific elements into the overall conceptual framework and research design. By contrast, in the ‘adapted etic approach’, theories and constructs are assumed to be universal. Then, following an examination of these assumptions in each context, an attempt is made to adapt the theory, conceptual framework and research design to each context, taking into account the differences identified. Drawing from existing approaches, Polsa 30 proposes an alternative approach, crossover-dialog, which seeks to create a dialog between different sources of knowledge.
From a different perspective, new procedures have been recently developed to address this emic-etic dilemma. In particular, de Jong et al 31 propose an integrated methodology that allows the inclusion of country-specific or emic items in standardized or etic scales. This procedure yields country-specific yet fully cross-nationally comparable marketing scales.
At the research design stage, one of the most important methodological issues is to establish construct equivalence to ensure that constructs, objects and other stimuli have the same meaning and significance in different contexts. 17 Construct equivalence is concerned with three distinct aspects: functional, conceptual and category equivalence: 19
Functional equivalence deals with whether the concepts, objects or behaviors being studied are equivalent across cultures or countries in terms of the function or the role they perform. For instance, a bicycle is considered a means of transport in the Netherlands but for recreational purposes in the United States. Conceptual equivalence is concerned with whether the same constructs, objects and other stimuli exist in different cultures or countries and are expressed in similar ways. For instance, individual values such as materialism or concepts such as ‘the self’ may vary from one country/culture to another. Finally, category equivalence relates to the question of whether the same classification scheme of objects or other stimuli can be employed across the different contexts of analysis. It includes a wide range of issues such as product category definitions, socio-demographic classes or occupational categories. For instance, a beer can be considered as a soft drink in the south of Europe and as an alcoholic beverage in the north.
To establish construct equivalence, preliminary research in each of the analyzed contexts is necessary. While this can be expensive and time consuming, problems at this stage are potentially the most damaging and irreversible. Exploratory and qualitative research are the best options to establish whether the constructs, products and objects investigated are conceptual and functionally equivalent. Similarly, an extensive review of both the extant domestic and country-specific literature should be undertaken. Finally, the use of multicultural research teams and international collaboration is widely advised. 12, 13, 32
At the research design stage, three additional issues need to be considered: translation, calibration and metric equivalence. Traditionally encompassed within the concept of measure equivalence, these issues are interrelated with construct equivalence as the measures involve the operational definition of the construct. 3, 19
Translation equivalence is concerned with the translation of the research instrument into another language so it can be understood by respondents in different countries and has the same meaning in each context. The goal of translation equivalence is commonality in understanding the instrument. Therefore, equivalence of meaning, rather than literal translation, is most important. 20, 33 Translation problems may arise from different causes. For instance, sometimes terms cannot be directly translated without losing their meaning, and sometimes a term does not exist in the other language. As discussed later, different translation techniques have been proposed to deal with this.
Calibration equivalence refers to the equivalence in monetary units, measures of weight, distance, volume, and other perceptual cues, such as color and shapes. For example, if the distance between two points is measured in kilometers in one country and miles in another, then questionnaire items relating to this measure should be converted. Therefore, it provides assurance that the units of measurement and other perceptual issues are comparable across populations.
Finally, two aspects have to be considered when determining metric equivalence: scalar equivalence and the equivalence of the scale or scoring procedure. Scalar equivalence refers to whether a score obtained through a certain scale in one country or culture has the same meaning and interpretation in another. As such, this type of equivalence implies that two individuals from different countries or cultures with the same value for a variable (for example, the same likelihood of purchase of a product) would give the same score on the same scale (for example, a value of 4 on a 5-point Likert scale). Scaling or scoring procedures refer to the use of equivalent scales or scores procedures in different contexts. Inconsistencies in this facet may arise from different levels of familiarity with scaling and scoring formats. Category rating scales are frequently used in survey research. 34 However, respondents across different countries may interpret the scoring formats and extreme categories inconsistently. 35 For instance, while in some countries 5- and 7-point Likert scales are common, in others 20-point scales are more often used. Similarly, the use of nonverbal scales requires detailed analysis to determine the degree of comparison between countries and cultures.
The main area for consideration about measure equivalence revolves around the translation equivalence. Different techniques have been proposed in the literature including direct translation, back-translation, parallel translation, decentering and committee approach. 3, 19, 36, 37 Space limitations permit only a brief review of these techniques. To overcome the problems of direct translation in which a bilingual translator simply translates an instrument from one language to another, researchers employ more sophisticated methods. The procedure most commonly suggested is back-translation. In this iterative method, a bilingual translator translates a research instrument into another language. Then, the instrument is translated back into the original language by a second independent bilingual translator. If discrepancies are noted in this process, corrections are made. This process can be repeated until equivalence is achieved. Owing to its focus on semantics, the resulting translations may lack naturalness and comprehensibility. In addition, it assumes an etic approach, which can be problematic as equivalent words or constructs in the other language may not exist. Parallel translation is a similar procedure, albeit using two translators with a greater emphasis on wording. Under this approach two translators independently translate the questionnaire. Then, translations are compared and modified until agreement is reached on a final version. Other procedures include the decentering approach. Research instruments are developed by collaborators in each culture. After an initial translation, this procedure allows changing words and phrases to provide greater accuracy. An alternative collaborative approach is the committee approach, where a committee of bilingual translators and experts discusses alternative versions of a questionnaire, the meaning of items, and so on. This approach starts with an initial translation, generally using the parallel translation approach with members of the team working independently. Modifications are made until consensus is reached. The cooperative effort between people with different areas of expertise working together is the main strength of this procedure.
Furthermore, pilot studies and pretest are recommended. 17 Qualitative (for example, feedback by monolingual, bilingual respondents and field staff, as well as focus groups) and quantitative tests (for example, based on item response theory, described later) can be used to test the comprehension, clarity and coverage of the instructions, the questionnaire, and so on. 37 Similarly, it is also recommended to avoid the use of sentences, terminologies and figures of speech which are typical of only one culture or country. 29
To guarantee calibration equivalence, researchers should independently check conversions of the different measurement units and other perceptual cues. 12 Finally, both types of metric equivalence can only be examined after data collection. Therefore, suggestions and recommendations will be presented in the analysis data section. In this stage, however, preliminary research conducted in each context may provide guidelines regarding the selection of scales, response patterns and measurement methods.
Another important issue at the design stage is the sampling design. Two main levels of sampling can be identified: sampling of cultures or countries (discussed in the problem definition section) and sampling of the individual respondents. This section will focus on the latter. Problems regarding sampling in this level fall into three areas: the choice of respondents, the conflict between comparability and representativeness and the sampling methods.
The choice of relevant respondents is a key issue in sampling, since these can vary across cultures or countries. For instance, women can be suitable respondents in some countries but not in others (that is, male-dominated societies). Similarly, senior managers may play a key role in the organizational decision-making process of Asian or Latin countries, whereas middle managers may have this role in Anglo-Saxon cultures. 17, 19 Another key issue concerns the conflict between the need for representativeness and comparability of the samples. 38 While homogenous samples enhance comparability and are needed to ensure equivalence, they are not likely to be representative of the target population. Therefore, balancing these two extremes represents one of the most important dilemmas in cross-cultural research. Finally, the use of probabilistic methods (for example, random and stratified sampling) enhances the likelihood of obtaining a representative sample. However, they are often not a viable choice. For instance, lists or directories are not usually available in emerging country markets. Therefore, in much cross-cultural research, non-probabilistic methods, such as quota sampling and judgmental sampling, are used. 39, 40 Researchers use these procedures to draw matched samples, that is, samples as similar as possible in terms of some relevant variables (sex, age, education, and so on). These procedures facilitate the control of extraneous variables that could potentially confound the results.
Hult et al 12 suggest enlisting parallel respondents for each unit of analysis. This can be useful to describe and compare their position, role and responsibility in relation to the subject under study in each country or culture of analysis. Based on the type of research conducted, Reynolds et al 38 propose a framework that provides interesting implications for the conflict of representativeness versus comparability, noted above. When the objective of the study is to examine attitudes and behaviors within specific countries or attributes of a cross-national group, representativeness of the country or specific population of interest is required. Thus, probability-sampling techniques are preferred. By contrast, when the objective of the study is to examine differences or similarities between cultures or countries and to examine the cross-national generalizability of a theory, model or construct, between-country comparability is the most important sampling objective. Therefore, non-probabilistic methods are preferred. Importantly, if matched samples are used to ensure comparability, the homogeneous samples selected should be suitable and relevant for the investigation. Similarly, the matching variables need to be relevant, logical and based on theory. In addition, researchers should be aware that this procedure may mask cultural differences and that results are limited to the specific groups analyzed. 29, 38
To enhance the comparability of the data collected, attention must be paid to the following aspects: equivalence of administration, equivalence of response, status and authority of the researcher and timing of data collection.
Equivalence of administration refers to the fact that the research settings and the instructions must be equivalent, not identical. 20 Special attention should be paid to physical, technical and social administration conditions. For example, whether a survey is administered individually or in groups could affect the results. Response equivalence is concerned with the design and administration of the research in such a way that people's responses to the questionnaire are equivalent on several dimensions, such as the respondent's familiarity with the test instruments, their levels of anxiety and other psychological reactions. 17, 20 One of the major concerns related to people's responses is the presence of response bias, which occurs when people's responses to a questionnaire are influenced by content-irrelevant factors. In this situation, the response does not indicate what it was intended to measure, threatening the validity of the findings seriously. In interviews, the status and authority of the researchers can also influence the results. This would include factors such as the characteristics of the interviewer (for example, affiliation, origin, gender, and so on) and the respondents’ confidence in the researcher. 29 For instance, the fact that the researcher is a foreigner may trigger unexpected reactions from respondents. Finally, the timing of data collection is also important. Data should be collected from different countries within acceptable time frames to enhance comparability. 16 Otherwise differences in factors such as the underlying economic and social situation may lead to different results.
The recommendations mainly focus on the adequate selection, training, supervision and evaluation of interviewers. Some authors suggest assigning interviewers randomly and recording their characteristics. The use of local agents is also advised. 13, 17 Data collection equivalence can also be accomplished by making the setting and instructions of the research equivalent, using uniform data collection procedures and collecting the data within acceptable time frames. Furthermore, it is recommended to carry out preliminary research to assess the equivalence of the modes of administration. If needed, variations in the data collection procedures (for example, personal interviews, mail surveys, and so on) are justified to achieve equivalence. 1, 12
Post-data collection assessment of equivalence has traditionally focused on measurement equivalence or invariance, which analyzes whether or not a measurement instrument yields accurate data about a specific issue across different groups. 41, 42, 43 Since Douglas and Craig 44 asked for evidence of measure equivalence, in general, and metric equivalence, in particular, numerous efforts have been made to refine this issue. Measurement equivalence or invariance (ME/I) concerns ‘whether or not, under different conditions of observing and studying phenomena, measurement operations yield measures of the same attribute’. 45 Accordingly, ME/I refers to the extent to which the content of each item is being perceived and interpreted in the same way across samples.
Researchers have proposed different procedures for establishing ME/I, which range from simple methods such as profile analysis, to other more sophisticated such as confirmatory factor analysis (CFA) and multigroup CFA, item response theory and generalizability theory. Multigroup CFA is one of the most recommended statistical approaches for assessing ME/I. Typically, frameworks suggested by Steenkamp and Baumgartner 46 and Vandenberg and Lance 47 are followed. This approach involves the use of hierarchical nested models to provide empirical evidence of the different levels of invariance (that is, configural invariance, metric invariance, scalar invariance, error variance invariance, factor variance invariance, factor covariance invariance and latent mean invariance). In their assessment, constraints are successively introduced in different parameters (for example, factor loadings in metric invariance; intercepts in scalar invariance). Then, the fit indexes of the least restrictive model are compared with those of the constrained invariance model. If full measurement invariance cannot be established, partial measurement invariance is examined. The level of invariance that should be satisfied depends on the goal of the study. For instance, configural invariance is sufficient if the objective is to explore the basic meaning and structure of the construct across countries or cultures, whereas metric and scalar invariance are additionally required if the purpose is to conduct comparisons of means.
Item response theory (IRT) is an alternative approach for assessing equivalence, although applications in marketing remain scarce. 48 This approach examines the extent of differential functioning of each item. That is, whether the individual items that comprise a scale function in the same way in each research context. IRT models, traditionally used with dichotomous items, have been extended to be used with items with multiple ordered response categories. Further, new developments based on this approach have been recently proposed. For instance, de Jong et al 49 present a model based on IRT to assess measurement invariance that solves some limitations of multigroup CFA, such as the requirement of at least partial invariance.
Generalizability theory (G theory) assesses to what extent a measure taken in a particular occasion can generalize to other measurement conditions. These measurement conditions, called facets, include different aspects such as countries, set of items, subjects, administration modes, time, and so on. The applications of G theory in the marketing literature are also scarce. Sharma and Weathers 50 were one of the first to use G theory to assess the cross-national applicability of a scale. More recently, Durvasula et al 51 extended this work and offered a procedure for conducting this analysis. Although G theory is less statistically rigorous than the CFA approach, it provides more information on the causes of variation across countries if the measure is found to be variant. That is, G theory allows the researcher to know whether the variation is due to the items, subjects, dimensions, countries, and so on. Furthermore, G theory is useful when multi-dimensional constructs are evaluated and large sample size requirements imposed by CFA are not met. 52
Interestingly, all the procedures commented above for assessing ME/I are suitable only for measures composed of reflective indicators, commonly used in practice. Reflective indicators denote manifestations of an underlying latent construct. That is, the main feature of these measures is that the direction of causality flows from the construct to the indicators. However, in many cases, indicators are causes of the construct rather than its effects. In other words, the direction of causality flows from the indicators to the construct. These indicators are known as formative. Filling this gap, Diamantopoulos and Papadopoulos 53 have recently outlined a procedure to test ME/I in formative measures. This procedure provides a basis to test the assessment of three types of ME/I conceptually consistent with the nature of formative measures: structure invariance, slope invariance and residual invariance. This framework involves three steps: testing for metric invariance of the reflective indicators added to solve the identification problems of formative measurement models; estimating a baseline multiple indicators – multiple causes (MIMIC) model; and introducing the equality constraints to test the different levels of measurement invariance.
Globalization has driven the need for reliable information about international markets. Gathering cross-cultural data is, however, not an easy task. Specific problems and pitfalls associated with cross-cultural studies have been the subject of inquiry for researchers across several fields (for example, psychology, management, education, sociology, and so on). Recent reviews, however, indicate that many cross-cultural studies still have major drawbacks due to the lack of methodological rigor. 11, 12 This article has presented a review of some of the most relevant methodological issues in the main stages of cross-cultural research. In addition, suggestions and recommendations based on the new trends and recent literature have been provided. The main recommendations are summarized in Table 1.
Table 1 Summary of general recommendations and suggestions identifiedAlthough we have included as many methodological issues as possible, not all of them have been addressed in this study. Therefore, we advocate future research to explore other issues, such as the presence and analysis of response bias, where important advances are taking place. Cross-cultural research is continuously progressing. Therefore, future research should also undertake a systematic examination of the methodological developments in this type of research to provide additional insights and a comprehensive and up-to-date review. Despite these limitations, it is hoped this article encourages greater methodological rigor in survey research conducted across cultures and/or countries. Only by paying attention to these issues will researchers and managers develop more theoretically robust and managerially relevant international market research.
The authors would like to thank the following sources for their financial help: I+D+I project (Ref: ECO2009-08283) from the Government of Spain and the project ‘GENERES’ (Ref: S-09) from the Government of Aragon and the European Social Fund.
