International Journal of Horticulture & Agriculture

The article describes an application of the Analytic Hierarchy Process (AHP) to evaluation of criteria for agro-industrial projects. For this purpose, a database was collected from interviews with experts in agricultural regions of Russia (South Urals) in 2015-2016. Experts are classified according to their relationship to agro-industrial business, work experience, education, and gender. The questionnaire was developed based on the analytic hierarchy process methodology. Part of the questions was based on the approach of the prospect theory in order to identify expert attitude to risk associated with the implementation of investment projects. In total, 226 experts responded to the questionnaire. Criteria of agro-industrial projects - financial, social, and risk criteria - were ranked according to their importance and taking into account the expert attitude to risk. Estimates of the criteria importance and expert features analyzed in the study may be used to improve the presentation of agro-industrial projects thus contributing to their successful evaluation.

Keywords: Agro-industrial projects; South Urals; Evaluation; Analytic hierarchy process; Risk;

Among the important characteristics of Agro-Industrial projects (AI projects) the researchers note the following: a) seasonality of production and supply of raw materials; b) perishable raw materials and products that require expensive facilities and storage costs; c) variability in crop yields from year to year; g) forming a herd (in a dairy industry) or plantation (in a fruit industry) takes up to 6 years [1]. Investment costs in AI projects in Russia, as in other countries of the Eurasian Customs Union, are high. For example, for 1 kg of production the cost of investment is as follows: $4.6 – tomatoes in winter greenhouses (Ural region, Russia), $4.9 – production of pigs in live weight in farms with a full production cycle (Belarus), $6.67 – turkeys in live weight in an integrated farm (Central Russia), and $37.5 – green vegetables in greenhouses (Moscow oblast) (data obtained during the work of one of the authors for Agro Top company, Israel). These factors explain a long-term return on investment and high risks of AI Projects in Russia [2, 3].

At the same time, AI projects have a positive impact on the development of the region and its financial and social indicators.In this study, the most important of these indicators include profitability, production risk, employment and migration indices, environmental risk, and social well-being. A variety of financial indicators and the impact of the implementation of AI projects on the social indicators in the region explain why multi-criteria evaluation of projects is important and needs taking into account: a) indicators of economic and social efficiency of the project, b) risks associated with its implementation, c) opinions of experts from various fields - project initiators, investors, bank managers, region leaders, researchers [4]. In the process of multi-criteria evaluation of projects, weight (importance) of criteria can be calculated based on expert judgments [5]. Profitability, payback period, risk, social benefits for the region - these and other criteria ranked according to their importance can be used in business discussions or in the form of multi-criteria formal procedures.

One way to assess the relative importance of criteria is the use of the Analytic Hierarchy Process (AHP) [6]. In this article, paired comparisons of criteria are made by experts from various sectors of agribusiness - from management and administration, finance, business, research and academic education. Next, the statistical processing (using AHP algorithms) of expert answers to questions, containing paired comparisons, allows to calculate the relative importance of the criteria. The consistency of the responses for each expert is taken into account. The evaluation of criteria can be structured by levels of the hierarchy, as it follows from the name of the method. In the presented study, we consider criteria (the first level of the hierarchy) and groups of criteria (the second level) [6, 7]. AHP and its generalizations are widely used for the evaluation of investment projects and the transfer of technology, and related socio-economic effects including the effects of the implementation of agricultural projects. In AHP experts can compare the importance of those criteria, which cannot be quantified. This is an advantage of the method which enables comparing qualitative criteria - social, environmental, and criteria of risk which are important when considering the agro-industrial business [8, 9]. The practical need to create a sufficiently large (several hundred respondents) data bases of expert judgment data can be explained by the following reasons. Firstly, when the number of respondents in each group of experts is large enough (greater than 50) limitations of their competence are less significant [10]. Secondly, to compare different groups of experts - let us say 3-4 groups of experts, and to meet the requirement of at least 50 experts in each group, the number of respondents in the database has to be greater than 150-200.

The aims of this article are as follows: (1) illustrate the possibility of using the developed questionnaire and collected database to calculate the importance of AI project criteria using AHP and other statistical methods; (2) determine the significance of the differences between importance of the criteria - financial, social, and risk criteria; (3) determine the significance of differences between importance of the criteria for expert groups, which differ in risk attitudes. This will enable a better understanding of multi-criteria evaluation of AI projects and improving project preparation for evaluation.

A sample of 226 experts was surveyed during the study conducted in the Republic of Bashkortostan and Orenburg oblast in Russia (South Urals) in 2015-16. Among the experts there were acting managers and agribusiness specialists, researchers, and senior students of economic departments of universities. All data were collected through personal interviews with respondents using a questionnaire developed by the authors.

In economic terms, the South Urals is a major agricultural centre of Russia. The main crops are winter rye, oats, flax, wheat, sunflowers, and sugar beet. The main livestock are cattle, sheep, and Orenburg goats. Every year several major projects are implemented in the region. Ethnographers note heterogeneity of the ethnic composition of the population of South Urals [11].

The database was established in MS Excel and included: (1) primary data from responses to the questionnaire, (2) calculated weights of each criterion for various groups of experts, and (3) calculated consistency indices of the responses for each of the experts. The data were analysed using Excel functions.

The questionnaire included 37 closed questions. The questions compiled brief definitions and descriptions for the interviewers to assist in their answers to possible questions of respondents. In the first questions experts compared in pairs the importance of the criteria separately for each of the three groups - financial, social, and risk criteria. The group of financial criteria included maximum net income, lowest payback period, and maximum return on investment. Questions about the risks related to technological risks (for example - yields, productivity, safety of cattle), the risk of reduction in product prices, and increasing investment costs of the project. Social criteria included the number of added work positions, possible reduction in retail food prices, and food security.

Then experts compared in pairs importance of ethnosocial and well-being indicators: public security - use of migrant workers, reducing ethnic tensions, reducing the risk of social conflicts and crime, environmental safety indicators (environment, health, labour protection), and social wellbeing of the population (material well-being, ethno-cultural needs, social infrastructure).

Next, in accordance with the principles of hierarchy adopted in AHP, questions of the second level were formulated. At this level, the experts compared in pairs the importance of groups of criteria: a) separately for three groups of criteria, financial, social, and risk, and b) on three groups of ethno-social and social wellbeing criteria - public security, environmental safety, and social well-being of the population.

Questions related to expert attitudes to risk (are they riskaverse or not) were formulated in accordance with the concepts of prospect theory. The answers to these questions enable determining whether certainty, reflection, and isolation effects are observed among selected experts. In particular, for testing the certainty effect the questionnaire included the following question:

Of the following two options, I would have chosen a project that brings profit:

answer A - 200 mln rouble with the probability 80%, or 0 rouble with the probability 20%,

answer B – 150 mln rouble with the probability 100%.

The questionnaire also included questions on sociodemographic characteristics of experts [11, 12].

The database was developed for input and initial analysis of the survey data. For every year, the module of data input includes 194 rows of questions and different options of answers. Of these, 152 rows relate to the questions on the importance of criteria, 17 – to the aforementioned effects of prospect theory, and 25 – to socio-demographic characteristics of experts. Number of columns in the module equals the number of respondents in this year. For each question, the answer selected by the expert is marked as “1” (Fig. 1).

Comparing any criteria A and B, an expert selects between the answers “A much less important than B”, “less important”, “equally important”, “more important” or “much more important”. Each option is assigned a numerical value according to one of the scales accepted in AHP. Denote importance of the criterion i with respect to the criterion j as aij , and suppose that for each criteria i, j the equalities a_ii = 1 and a _ji = 1/ a_ij are valid. Then answers of each expert are written in the form of a positive reciprocal symmetric matrix of judgments [6, 7] (Fig. 2).

All matrices of adjustment (four for each of 226 experts) were tested for consistency of expert answers. For example, if the expert answers that criterion a is more important than criterion b, b is more important than c, and c is important than a, his answers are considered inconsistent. Saaty suggested the following index of consistency:

CI = (λ_max −n)/ (n−1)

Where n is the number of criteria (three in this study),

λ_max is the main Eigen value of the matrix of judgments for this expert.

Saaty suggested comparing CI with the same index averaged over a large number of positive reciprocal symmetric matrices of the same order whose elements were obtained randomly («coefficient of consistency»). For n = 3 this average index equals 0.58. Saaty recommended using answers of experts for whom the ratio of CI to the value 0.58 less or equal 10% [6].

Analytical modules of the database modules enable the following calculations: matrices of judgements, importance of criteria (groups of criteria), consistency indices of the responses. The calculations were performed for single experts and for experts grouped by socio-demographic and psychological (in particular, by risk attitudes) characteristics. Additional details needed for performing these calculations and a literature survey of AHP as one of experimental economics methods are available in [13].

All calculations were made in the database in MS Excel 2013. In this study we used the balanced scale of assigning the weights to expert answers [14]. For the case of five possible answers, as in our study, this scale is presented in Table 1.

The calculation of matrices of judgements enabled to change the values “1” in the expert answers (Fig. 1) to the values from Table 1. Then eigenvalues of matrices of judgements were calculated and on this bases - coefficients of consistency of expert answers [6].

Importance of criteria was calculated, first, for every criterion for single experts. For example, for the group of financial criteria

the relative importance in % of “net income”, “payback period”, and “return on investment” was calculated. For the group of risk criteria - the importance of “technological risk”, “reduction in product prices”, and “increasing investment costs” was calculated, and so on. Then the importance of the criteria was averaged over all experts (hereafter “importance averaged over experts”).

In the second method, which was used for comparing the importance of criteria, we calculated the number of the first places (or the first/second places if two criteria appeared to be the most important), awarded by single experts (“importance by first places”). For each expert the first place (the most important criterion) was determined using the main eigenvalue of matrix of judgment, as it is usual for AHP [6]

Statistical analysis of differences in criteria importance was tested using Z-test at the 5% significance level. For each group (consisted of three criteria) the significance of difference in importance between the criteria 1 and 2, 1 and 3, and 2 and 3 was tested.

70-75% of respondents answered the questions on occupation and education. The highest share of experts was university students (31%), 26% worked in agro-industrial enterprises, followed by researchers and academic teachers - 20%. More than half of the experts, 59%, had a degree in economics, and 78% of all experts had any higher education. Two-thirds of the experts were women. The average work experience in the expert occupation was 8.3 years (Fig. 3).

Those experts, for which coefficients of consistency in several groups of responses were greater than 10-15% [6] or all of the answers were chosen as “equally important”, were excluded from the analysis. The number of such experts was about 15% of all respondents.

95% confidence intervals were calculated for the importance of every criterion. Financial criteria, for example, were ranked as follows: “net income” received the 1st place, “payback period” - the 2nd place, and “return on investment” - the last 3d place. Importance of criteria evaluated by the number of the first places was calculated twice: for all experts and for risk-averse experts. For the latter, technological risk and risk of reduction in product prices were much more important than the risk of increasing investment costs. For results, calculated for all the experts, such a distinction was not observed. Results of evaluation of the importance of the criteria in all three groups - financial, social, and risk criteria - are set out in Table 2.

Relative importance of three groups of criteria included in the questionnaire - financial, social, and risk criteria - was analyzed. For “importance averaged over experts” the difference between the groups of criteria was not significant. However, “importance by first places” calculated for all experts is significantly different than that calculated for risk-averse experts. The calculation for all experts: the importance of each of the groups of criteria is in the range 31-35% (calculated by the data from column B, Table 3). The calculation for risk-averse experts: the importance of the financial criteria was assessed as 40% and of social and risk criteria - 30% for each of these two groups (calculated by the data from column C, Table 3).

These results indicate the possibility of using the questionnaire developed in this study for evaluation of criteria for

agro-industrial projects. The completeness of the questionnaire and sufficient size of the collected database enabled evaluation of differences in the importance of criteria - between groups of criteria and between groups of experts.

This study provides a database containing expert evaluation of criteria for agro-industrial projects. We developed a questionnaire than includes 37 questions to experts from different groups: project initiators, investors, bank managers, region leaders, researchers and senior students. Total 226 experts from cities and agricultural regions responded the questions. The database includes about 8,400 answers to the survey questions.

The database structure was designed based on the questionnaire and the research questions of the present study: determine the importance of AI project criteria and the significance of differences between them - for single criteria, groups of criteria, and groups of experts. The suggested questionnaire and the size of the collected database enabled answering these questions. In particular, the opportunity to distinguish between experts that differ in their attitude to risk (“certainty effect”) was studied.

One of the features of AHP used in this study is the ability to use experts with diverse experience and education, of different age and sex. The different importance they attach to various criteria reflects their different social views [16], part II, section 5. In this sense, the work with the suggested in this study database using AHP enables integrating views of social groups that hold different interests.

The results shown in Tables 2 and 3 were obtained in two ways. In the first of them (column A in each of these tables) importance of criteria is averaged over experts as it is usually performed in AHP. In columns B and C the most important criteria are those that were received the maximal number of first places awarded by experts. This method (“importance by first places”) is a simple analogy of the method of voting in political elections (methods of Hill, Hare as classified and described in [17]). The combination of using AHP for evaluating the importance of alternatives (criteria in this study) on the one hand, and using voting procedures for selecting the best of these alternatives, on the other hand, is used in various applications. In the study [18] this approach was used to rank by importance widely used sources of information on the Internet, when 14 doctoral students were experts.

The results shown in Table 3 are interesting because they enable comparing the criteria importance averaged over all the experts and over those of them who are risk-averse. The importance of risk factors has not changed much: 55 and 39 experts that awarded the first place to this group of criteria are, respectively, 30.6% and 30.0% of all experts related to “all experts” and “risk-averse experts”. However, the importance of the group “social criteria” has decreased significantly: 62 и 39 experts составляют 34.4% и 30.0%. Accordingly, the importance of the group “financial criteria” increased: from 35.0% (63 experts) in column B to 40.0% (52 risk-averse experts in column C).

The explanation for this phenomenon may be found in the published research on behavior of Socially Responsible Investors (“social sustainability,” “green,” “ethical”) investors (SRI). It is known that so-called “ethical investors” have ancient roots in the Judaic, Christian, and Muslim traditions that practice a number of relevant social norms [19]. In the study [20] it is found that SRI is less concerned about the return on investment and, consequently, they are less risk- averse. These results may explain why among risk-averse experts for which, as it is discussed above, the share of SRI may be lower than among all experts, the importance of financial criteria increases and the importance of social criteria decreases.

The practical aspects of the study lie in the applicability of the developed methodology and in using estimates of the criteria importance when discussing the benefits and risks of AI projects. Expert features analysed in this study may be used to improve the presentation of AI projects thus contributing to their successful evaluation.

The study was supported by the Russian Foundation for Humanities grant under the project No.15-02-00044 “Ethnoconfessional and socio-economic aspects of the region’s investment strategy: the development and validation of multicriteria evaluation method agro-industrial projects in the South Urals”.

The authors are grateful to Irina Frolova, prorector of the Bashkir Academy of Public Administration and Management; Vasilii Lukyanov, dean of the Department of Economics in Bashkir State Agriculture University; Venera Muratova, Deputy Director of the Institute of Economics, Finance and Business in Bashkir State University; Airat Baimov, Regina Muchametzyanov - Duggal, Timour Nadirshin, Ainur Tuzbekov, Aigul Shagieva, researchers in R.G. Kuzeev Institute of ethnological studies, Ufa Research Centre of the Russian Academy of Sciences - for their contribution to organizing and conducting of the survey.

1. Austin JE. Agroindustrial project analysis. The Johns Hopkins University Press. 1992.
2. Arkhipova AS, Svetlov NM. Mathematical simulation in management of investment attraction of agro industrial complex. Scientific Journal of KubSAU. 2012;76:1-11(in Russian).
3. Goncharenko LP, Geraschenkova TM. Innovation and investment processes in agricultural production as a factor to improve food security in Russia. Vestnik finansovogo universiteta. 2014;2:13-23 (in Russian).
4. Yom Din G. Performance and profit sensitivity to risk: a practical evaluation of the agro-industrial projects developed by Israeli companies for the CIS and Eastern European countries. Agr. And Food Economics. 2013;1(1):1-23.
5. Bjørndal T, Herrero I, Newman A, Romero C, Weintraub A. Operations research in the natural resource industry. Int. Transactions in Operational Research. 2012;19(1-2):39-62.
6. Saaty T L. A scaling method for priorities in hierarchical structures. J Math Psychol.1977;15(3):234–281.
7. Saaty TL. How to make a decision: the analytic hierarchy process. European J of Operational Research.1990;48(1):9-26.
8. Zabihi H, Ahmad A, Vogeler I, Said MN, Golmohammadi M, Golein B, Nilashi M. Land suitability procedure for sustainable citrus planning using the application of the analytical network process approach and GIS. Computers and Electronics in Agriculture. 2015;117:114-126.
9. Sahnoun H, Serbaji MM, Karray B. Medhioub K. GIS and multi-criteria analysis to select potential sites of agro-industrial complex. Environmental Earth Sciences. 2012;66(8):2477-2489.
10. Tsyganok VV, Kadenko SV, Andriichuk OV.  Significance of expert competence consideration in group decision making using AHP. Int J of Production Res. 2012;50(17):4785-4792. DOI: 10.1080/00207543.2012.657967.
11. Yunusova AB, Yom Din G, Tuzbekov AI, Nadyrshin TM, Mukhametzyanova-Duggal RM, Baimov AG. Ethnosocial factors of realization of investment strategy in the South Ural. Isvestiya UNC RAN [Proceeding of the RAS Ufa Scientific Cenre]. 2015;4:87–105 (in Russian).   
12. Yom Din G, Yunusova AB. Evaluation of investment agro-industrial projects: criteria and risks. Econ. Analysis: Theory and Practice. 2016;7(454):4–17 (in Russian).
13. Ishizaka A, Balkenborg D, Kaplan T. Does AHP help us make a choice & quest; An experimental evaluation. J of the Operational Research Society. 2011;62(10): 1801-1812.
14. Salo AA, Hämäläinen RP. On the measurement of preferences in the analytic hierarchy process. J of Multi-Criteria Decision Analysis. 1997;6(6):309-319.
15. Brunelli M. Introduction to the Analytic Hierarchy Process. 2014. Springer.
16. De Montis A, De Toro P, Droste-Franke B, Omann I Stagl S. Assessing the quality of different MCDA methods. Alternatives for environmental valuation. 2004;4:p.99.
17. Volsky VT. Prozeduri golosovaniya v malih gruppah. Control sciences. 2016;2:2–23 (in Russian).
18. Srdjevic B, Pipan M, Srdjevic Z, Blagojevic B, Zoranovic T. Virtually combining the analytical hierarchy process and voting methods in order to make group decisions. Universal Access in the Information Society. 2015;14(2):231-245. DOI: 10.1007/s10209-013-0337-9.
19. Chow GW, Durand RB, Koh S. Are ethical investments good? Australian J of Management. 2014, p.0312896213516327.
20. Pérez-Gladish B, Benson K, Faff R. Profiling socially responsible investors: Australian evidence. Australian Journal of Management. 2012;37(2):189-209.
21. Yerasov IV 2014. Instrumental methods of the economy. (66) Wax, 6/2014.
22. Klimov VA, Shatohin MV, AA Chernikov, Duplin VV. Financial estimation of regional agrarian and industrial complex. Bulletin of the Kursk State Agricultural Academy. 2014;7.
23. Renneboog L, Ter Horst J, Zhang C. Socially responsible investments: Institutional aspects, performance, and investor behavior. Journal of Banking & Finance. 2008;32(9):1723-1742.