Social policy research methology
Опубликовано на портале: 14-09-2011
Interviews stood alongside statistical data as an important method of obtaining information for the project. The interviews were with members of "labour market crises groups" who represented different forms or stages of unemployment (people on compulsory unpaid leave or short-time working at the instigation of management, people under notice of redundancy, registered unemployed and people who had found work after a period of registered unemployment). The significance of this source of information arose primarily out of the unreliability of official statistics, coupled with the opportunity the interviews provided to gain a deeper understanding of the socio-economic phenomena we were studying.Each of the two possible ways of obtaining information through interviews (relatively short questionnaires with largely closed questions, used for large representative samples, and in-depth qualitative interviews with small samples and a large number of open questions) has its advantages and disadvantages, and the choice between them was governed in the end by the subject matter and aims of the research in question.In our case, we had to make sense of the cause-and-effect relationships within a very serious cluster of problems, where macro-level problems (for example, the structural reorganization of industry, the crisis taking place in a number of sectors or the different pace at which reforms are taking place in different regions) were superimposed on medium-level and even on micro-level problems (household structure of the unemployed, their state of health, etc.). This could not be done using a short questionnaire and quantitative methods of analysis. We therefore preferred the in-depth interview method. The main drawback to this method is its lack of representativeness. Use of in-depth interviews enables the existence of problems and relationships of one kind or another to be established, but does not permit a reliable evaluation of the scale and extent of the various positions recorded. In our case, however, where the main task was to understand the essence of the various phenomena and the causation underlying them, rather than their extent, this drawback could be ignored.The difficulty of using in-depth interviews with small samples in the course of our research actually lays elsewhere. The research objectives involved not only comparing and analyzing the interaction between macro- and micro-level factors, which made the questionnaire extremely long, but also comparing the situations in different regions on the one hand, and in respondent groups with differing employment status on the other. This meant that, with three regions and four separate groups, we had to conduct an independent analysis of 12 relatively independent research subjects, the sample for each of which, even using in-depth interviews, could not be smaller than 15-20 people. The total number of respondents therefore had to be at least 200.These considerations caused major problems in processing the data obtained, and made it necessary, in addition, to employ closed questions, an element of large-sample research, in our research. The assumption was also made that our groups might include people experiencing the most severe socio-economic deprivation; and experience of other sociological research has shown that this contingent often has difficulty in expressing its position, especially with open questions.In drawing up the questionnaire, therefore, the specific nature of the subjects of our research was taken into account to the maximum possible extent, and the majority of the questions were of a closed type, although in fact this did not rule out the addition of any opinions not considered when the "list" of questions and variants of responses to them was being drawn up. Consequently, the "closed" nature of the questions was relative and, as it were, suggestive, accompanied as it was by an invitation to the interviewee to voice some other opinion. Nevertheless, with two phases of research, the total number of open questions amounted to several dozen. Most of these were subsequently coded, while some were subjected to content analysis. Coding was also applied to some of the responses to questions which invited the respondent to express an opinion over and above the response variables, although in the event there were very few people who did wish to express an additional opinion.The use of a large number of closed questions in in-depth interviews naturally meant that the closed questions themselves had to be of high quality. Almost all the closed questions were therefore taken from interview questionnaires that had already been used during other studies and had worked well, both in the way they encompassed all possible respondent positions, and as regards the importance of the results obtained. We took as a basis an interview carried out in 1994 as part of a representative study of the Russian labour market, run by one of the present authors. This was supplemented by questions from the research armory of Western (Cohn, Rose, Townsend, Dean, Taylor-Gooby) and Russian (Chemina, Lapin, Magun, Tikhonova) sociologists, and from monitoring surveys carried out by RN1SINP and VTSIOM. Besides the quality of the closed questions used, this approach to creating a research armory ensured comparability of the results obtained from a survey of Russian unemployed with data from in-depth interviews with unemployed people in Europe, on the one hand, and of data for members of critical groups being investigated in the labour market with data for other Russians, on the other hand.Use of an in-depth interview method does not, of course, allow the results obtained to be extrapolated to embrace the entire Russian unemployment picture. But it did allow a quite large body of data (480 interviews with 274 people, taking into account substitutions and the two stages of the research) to be used to capture certain phenomena and trends which it was not possible to record using either statistical methods or methods of quantitative representative research or the classical in-depth interview method with genuinely small samples.Our survey of policymakers played significant part in the research. This method used an approach in which the respondent is asked to give information on a range of phenomena about which he is well-informed and which are suggested by the survey organizers. (See: Methods of Information-Gathering in Sociological Research. Book 1. The Sociological Survey. Edited by V. G Andreenkov and 0. M. Maslova. Moscow, 1990, p. 24). Generally, this technique presupposes that people are selected on the basis of their formal professional status, of the results of testing their knowledge, and also of attestations from colleagues, to provide a sample of suitable policymakers. In this case, the numbers and representativeness of the group of policymakers were assessed not so much statistically as in terms of quality indicators, and this subsequently made it possible for the policymaker to give free expression to his opinions and to arguments supporting them. Open question types are dominant in this type of survey, with closed questions intended solely to evaluate the level of confidence of the individual, and the degree of agreement or disagreement with the positions of other specialists which have already been voiced.Unfortunately, this method of obtaining information does not by any means always provide the required results, since the accuracy of the data obtained is limited to the framework of the personal practical experience of the policymakers, and to their professional and even their personal interests. It is also almost impossible to avoid the "traps" of interviewing techniques for use with experts, that are described in the literature. In the first place, the sampling of policymakers on the basis of formal characteristics (professional status) narrows the range of suitable informants; in the second place, the same principle brings unsuitable people into the sample; in the third place, the questions put are often within the frame of the researchers' interests, but are not within the competence of the respondents (this happens particularly frequently when the questions are drawn up by representatives of other socially, ethnically or culturally alien cultures). (On this question, see: S. A. Belanovskii. Metodika i tekhnika fokusirovannogo intervyu [Methodology and techniques of focused interviewing]. Moscow, 1993, p. 245-249).In our case, these well-known faults in surveys of experts were borne out with particular emphasis. This is because all Russian officials, businessmen, managers and trade union bosses were and still are neophytes in the world of the market, and neophytes in the new institutional structures which came into being 5 or 6 years ago. To improve the reliability of our results, we increased the size of the policymaker sample by comparison with the customary number, we repeated the interviews with them after a year, and we bore in mind throughout the results of our parallel survey of members of the weak social groups who were 'being looked after by" our policymakers.In drafting the interview questions, we attempted to reflect the objectives of the research to the greatest possible extent, by emphasizing above all the importance of the policymakers' personal opinion in the questions. Although most of the questions were repeated in the second survey, some others were then asked for the first time. The inclusion of new questions in the second survey enabled us not only to uncover changes in the processes which had been examined by us and by the policymakers in 1996, but also to detect those processes and phenomena which had either not been accounted for at that time, or were not happening a year previously. It was also interesting to find out how the policymakers' attitudes to current events and their assessments had changed, and what changes there had been in the degree to which they were informed about, interested in and involved in tackling social problems.In conducting the interviews, we followed the customary method of carrying out developing a survey of experts.Furthermore, in most cases the interviewers, while observing the obligation to discuss the framework questions, did not adhere strictly to the interview schedule, which we feel was beneficial in that it turned the conversations into something like confidential discussions; this made for a considerable contrast with equivalent interviews in the Soviet era.In many cases, the interviewer was unable to achieve real openness on the part of the policymakers. This was most manifest in the questions about the respondents' social contacts. The respondents showed excessive caution in replying to these questions. It could be said that this fact bears witness to the peculiar nature of the socio-cultural environment which had grown up in elite groups in Russian society in Soviet times, and which will, it seems, not be readily overturned.We were struck by the way in which, in cases where good emotional contact was established with the interviewee, qualitatively new information was recorded during the interview. This was partly because, in most cases, the same respondents were interviewed twice, which disposed the respondent to be more open at the second meeting. Conducting in-depth interviews in a free form enabled interviewees to say their piece on the issues that were most important to them, not only (and not so much) on the issues which were a priory important to the researchers.One result of conducting interviews in a free form was that emotionally tinted information was obtained, information that was uppermost in the respondent's mind. Another result of using a free interview form was a build-up of excessive and contradictory information, which made analysis very difficult. Consequently, it was necessary to do more during the interviews than just obtain information on an item of interest; some assessment of the reliability of the information had to be made, with some kind of suggestion as to the angle of spin on it and some kind of explanation as to why the respondent was providing the information he was, whether it was of personal significance to him, etc. The information obtained from a free-form in-depth interview enables particular types to be identified among the respondents, these types being the bearers of certain socio-cultural characteristics; it also helps in understanding key figures, and to reveal the particular perception in one or another group of respondents. In most cases, the respondents were happy with the proposed discussion style: for many, it was a chance to be heard, to share pressing and painful problems with an interested and understanding listener. The text of the interview was transcribed verbatim, in "question-and-answer" form, retaining the respondent's own vocabulary.If we were to draw a preliminary conclusion, it would be that the interviews we gathered justified our fairly skeptical attitude to official documents, and gave us a more honest evaluation of the situation around social issues, reflecting the positions of actual or potential subjects of social policy in Russia. The policymaker respondents selected, who were specialists in social policy and employment policy, represented social groups from the Russian elite whose activities are closest to an understanding of these very important issues.