Keywords: Human-Automation Interaction; Production; Assembly; Digitalization
The aim of this paper is to present a review of the development of Human-Automation Interaction in order to see the trends of interaction between humans and automation and not specifically the technology used. This will be used in order to suggest cognitive automation strategies to support operator 4.0 and to find solutions that could increase operators' performance and satisfaction in a final assembly context.
Formulating strategies form cognitive automation is becoming more and more important since the complexity of the work environment increases. Findings from the literature review described in this paper shows that there is a gap within the research of measuring how the operators perceives this complex environment and what factors to measure to capture this. A lot of the research done within HAI is in the environment of controlrooms and air-traffic control, but there is not a lot done within the area of manufacturing and production. Technologies and automation has been in focus the resent years but it is seen that research within interaction and human factors is needed to fully understand the potential of HAI.
Today there are a lot of technologies available on the market in terms of tablets, glasses, Augmented Reality (AR), Virtual Reality (VR) etc. that could support the operator but there is not a lot of research on how the operator perceive these solutions and how the support the operators cognitive processes. The development of the tools must go from technology centred to human centred design in order to fit the capability of the operators [6]. Most research on cognitive automation in manufacturing systems focuses on parameters such as time, cost, flexibility and quality in manual tasks [34] and safety and collaboration in more automated tasks [35] not on operators' cognitive situation. One important challenge connected to this is to handle demands regarding social sustainability that makes it important to be attractive to a workforce with varying age, skills and health issues [36]. Handling this is connected to improving the operator performance that is to decrease process errors, achieve high quality, achieve good working conditions, fast processes, quick change-over's, and, to decrease cost [37,38]. During recent years much has been written in the Human-Automation Interaction area and especially focus have been on the area of autonomous cars. The focus connects to safety and control issues due to that the driver is not in the loop i.e. is not active in the decisionmaking and cannot take over the driving if the automation fails [39-41]. Complex interactions have therefore been investigated in terms of adaptive automation and decision-making [42-44]. Similarly much research is carried out to further understand the conflicting scenarios that can arise when a pilot does not see what the autopilot is doing (exemplifying the importance of visualising the system process of the automation) [45,46].
The evolution of digital tools [47,48], increased number of co-bots [8] and human-robot collaboration [49,50] in final assembly points towards an increased use of both cognitive and physical automation in the final assembly context. HAI will be more and more common and the need for structure and measure parameters related to HAI will be essential [19,51,52]. In the concept of operator 4.0 a smart and skilled operator performs 'work aided' by machines if and as needed. It represents a new design and engineering philosophy for adaptive production systems where the focus is on treating automation as a further enhancement of the human's cognitive capabilities [53].
Type of article: Review based journal papers from different
1 |
DEFINE |
1.1 |
Define the criteria for inclusion |
1.2 |
Identify the fields of research |
1.3 |
Determine the appropriate source |
1.4 |
Decide the specific search terms |
2 |
SEARCH |
2.1 |
Search |
3 |
SELECT |
3.1 |
Refine the sample |
4 |
ANALYZE |
4.1 |
Open coding |
4.2 |
Axial coding |
4.3 |
Selective Coding |
5 |
PRESENT |
5.1 |
Represent and structure the content |
5.2 |
Structure the article |
Citations: There are different criteria depending on the time span. Higher number of citations >80 for the articles before 2011 and >5 for the articles in the second time span.
Identify the fields of research: A lot of the HAI research is done within computer science or AI so the collection tries to be as broad as possible including as many research fields as possible.
Determine appropriate sources: The first search on google on Human automation interaction, gave over one million hits so the topic is clearly written about, a lot of the hits were not scientific so for the literature review a database collection were made as a first step in order to find scientific papers. Since the area of production is seen as an applied research field it is hard to choose the more theoretical data-bases such as web of science, nor a data base were the author has too much freedom to add papers themselves, such as Scopus. Therefore, two different databases were chosen; one broader and one more specific scientific database were collected: Google Scholar and Science Direct.
Decide on the specific search term(s): "Human-Automation Interaction" AND "complexity", "Human-Automation Interaction" AND "complexity" AND "production systems".
Stage 2 included articles written 2011-2014 where 40 Google scholar (366 papers found) and 23 science direct articles (23 found). The sample was captured this way to find the most cited articles from 2000-2010 and from 2011-2014. In total 25 papers were removed from the sample due to that they were doubles (56%), too extensive (32%) or due to that they did not fit the scope of investigation (12%). The key words were chosen this way because an expectation of a wider scope of articles due to research fields in the first search and a more specific production related scope in the second search.
Type of article stated if it was a case study, experiment, literature study or a theoretical paper. The result in figure 2 shows that there are a lot of different types of papers within this research area.
Figure 3 shows that this is a complex topic that is included in many research areas and research fields. The research area was found by studying the introduction or the keywords. 77 explicitly stated areas were found (4 papers had more than one area connected to it). Type of field was added to further state what type of field the article addressed; 88 fields were found
• The Human centered category incorporates factors used to study or describe human factors.
• The Automation centered category represents performance indicators used to describe automation systems for instance performance, error/failure management, cost/economy, changes etc. From a production perspective factors connected to performance and automation are very important since they represent the way the system is described in terms of its productivity, efficiency and flexibility.
• The Interaction centered category represents factors that are connected to the joint system of human-automation.
In total 690 sub categories were found. Most sub-categories were found in the Human centred (N = 260) and Automation centred category (N = 240) and fewer were found in the Interaction centred (N = 187) category. The sub-categories captured a wide range of fields, research areas and types of studies and could be seen to reflect recent HAI research. Some authors were represented more than others, which could have skewed the result. For instance the aeronautics field was represented by many similar authors. In aeronautics Levels of Automation is an important focus and also much focus lies in cognitive tasks and situation awareness. Independent of this, the issue of understanding complex tasks for supervisory control or for solving more cognitively demanding tasks is not yet solved. Also it is recognized that the sample cannot account for the full view of the interaction between humans and automation but that the structure performed is a first step towards a framework that can help design future complex systems. The most frequent subcategories (represented more than seven times in articles) are presented in Table 2.
In a production context performance, LoA, safety, reliability, control, function allocation and flexibility are common factors but mental workload, situation awareness, decision-making and trust are not among the most common studied factors. This indicates that more research is needed in the production area, and especially in complex assembly. This gap was also indicated in Mattsson, et al. [57] where it was indicated that operators working at complex stations need better cognitive support to reduce cognitive load.
The final step in the analysis is to put the sub-categories into a system context, also done in Mattsson, et al. [56]). One way to describe the system is by using the black box theory [58], with stimuli – black box - response. In this context, the stimuli could be described as existing conditions that could be changed i.e. design conditions. Examples of sub-categories are; levels of automation, layout, level of flexibility, operator skill, operational structure etc. These are the changeable parameters in the cognitive automation strategy. Inside the black box are the operators´ cognitive processes i.e. the mental processes in which humans become aware of and process information [59]. The insight of the black box is often described as very hard to describe, cognitive processes is even harder to describe sine every human thinks in different ways but there are several studies that has been done regarding opening the black box of cognitive theory [60, 61] but not in a production context.
The response or output in a production context are the operators´ performance with help of the input transformed within the black box into design results. Examples of sub-categories are quality, workload, trust in automation etc.
In Table 3 the number of factors in total and times in articles in total were divided according to key category and black box theory. By dividing sub-categories both according to categories and black box elements it was possible to see indications of both were much research have been focused and where more research is needed in order to formulate a cognitive automation strategy to support the operators´ cognitive processes.
The most interesting finding seen from a gap analysis is where there exist fewer (below average) sub-categories in total or per times represented in articles. The Human input
Category |
Most frequent subcategories |
Represented in number of papers |
Automation-centred |
Performance(operational) |
40 |
LoA* |
29 |
|
function allocation |
10 |
|
flexibility |
7 |
|
Human-centred |
trust/automation reliance |
38 |
mental workload |
24 |
|
SA |
21 |
|
decision making |
18 |
|
human errors |
9 |
|
Interaction-centred |
safety |
15 |
control |
11 |
|
design |
7 |
Key category |
Input to the system |
Black box |
Output from system |
Average per |
||||
Design |
conditions |
Cognitive |
processes |
Design |
results |
key |
category |
|
Number of sub- categories (total) |
Number of times in articles (total) |
Number of sub- categories (total) |
Number of times in articles (total) |
Number of sub- categories (total) |
Number of times in articles (total) |
Number of sub- categories (total) |
Number of times in articles (total) |
|
Human |
4 |
50 |
9 |
63 |
8 |
93 |
7 (21) |
69 (206) |
Automation |
11 |
83 |
5 |
12 |
10 |
79 |
9 (26) |
58 (174) |
Interaction |
7 |
33 |
6 |
42 |
10 |
40 |
8 (23) |
38 (117) |
Total number of sub- categories in black box |
22 |
166 |
20 |
117 |
28 |
212 |
||
Also Automation within cognitive processes were few. The analysis included the following sub-categories: fault diagnosis (diversity), planning, procedures and system behaviour. To be able to monitor the automation´s system processes could be important in order for the operator to better stay in the loop.
The result in table 3 indicates that more research is needed within cognitive processes and foremost automation of cognitive processes i.e. cognitive automation. Furthermore there is also a gap in what design conditions connected to the operators' needs and how these conditions could be measured and changed. By choosing a cognitive automation strategy best adapted to the operator and implementing it according to sociotechnical perspectives it is believed that the complex system will benefit the most e.g. it can handle uncertainties [65-67]. The system could in that sense be adapted to fit the operators' individual characteristics. As stated by Fredrick Winslow Taylor best efficiency could be reached if a person works according to his/her natural abilities [68]. This will benefit both social sustainability challenges as well as the demographical ones. Since one of the upcoming challenges regard handling digitalization, it is important that the system's transparency is increased [69]. This should include increasing situational awareness and communication, which is necessary for a dynamic function allocation to work. This could result in increased performance, control, trust and safety but also a decreased mental workload for the operator in theory. Figure 4 illustrates some solutions that support cognitive processes for the operators.
- Lotter B, Wiendahl HP, ElMaraghy HA. Changeable and Reconfigurable Assembly Systems Changeable and Reconfigurable Manufacturing Systems. 2009;127-142.
- Fasth A, Stahre J, and Dencker K. Level of automation analysis in manufactoring systems, 2010;233–242.
- Michalos G, Makris S, Papakostas N, Mourtzis D and Chryssolouris G. Automotive assembly technologies review: challenges and outlook for a flexible and adaptive approach. 2010:2(2);81-91.
- Fast-Berglund A, Fässberg T, Hellman F, Davidsson A and Stahre J. Relations between complexity, quality and cognitive automation in mixed-model assembly. 2013;32(3):449-455.
- ElMaraghy H and ElMaraghy W. Smart Adaptable Assembly Systems. 2016;44:4-13.
- Endsley MR. Designing for situation awareness: An approach to user-centered design: CRC press. 2016;396.
- Fast-Berglund A, Palmkvist F, Nyqvist P, Ekered S and Akerman M. Evaluating Cobots for Final Assembly. 2016:44;175-180.
- Tsarouchi P, Spiliotopoulos J, Michalos G, Koukas S, Athanasatos A, Makris S, et al. A Decision Making Framework for Human Robot Collaborative Workplace Generation. 2016: 44;228-232.
- Mattsson S, Tarrar M and Fast-Berglund Å. Perceived production complexity – understanding more than parts of a system. 2016;6008-6016.
- ElMaraghy W, ElMaraghy H, Tomiyama T and Monostori L. Complexity in engineering design and manufacturing. 2012:61(2);793-814.
- Schleich H, J. S. J and Scavarda LF. Managing Complexity in Automtive Production. 2007.
- Heilala J and Voho P. Modular Reconfigurable Flexible Final Assembly Systems. 2001:21(1);20-28.
- Fasth‐Berglund Å and Stahre J. Cognitive automation strategy for reconfigurable and sustainable assembly systems. 2013:33(3);294-303.
- Makris S, Karagiannis P, Koukas S and Matthaiakis AS. Augmented reality system for operator support in human–robot collaborative assembly. 2016:65(1);61-64.
- Putz M, apakostas N, Pintzos G, Matsas M and Chryssolouris G. Knowledge-enabled Design of Cooperating Robots Assembly Cells. 2014:23;165-170.
- Billings C. Aviation Automation: The search for a Human-centered approach. 1997; 337-348.
- Matthias Heesen, Tobias Hesse, Anna Schieben, Claas Brunken, Christian Löper, Johann Kelsch, et al. Interaction design of automatic steering for collision avoidance: challenges and potentials of driver decoupling. 2015:9(1);95-104.
- Johannes Weyer RDF, Fabian Adelt. Human–machine cooperation in smart cars. An empirical investigation of the loss-of-control thesis. 2015:72;199-208.
- Dehais F, Peysakhovich V, Scannella S, Fongue J and Gateau , "Automation Surprise" in Aviation: Real-Time Solutions. 2015;2525-2534.
- Oliver Straeter MA and Marcus Arenius. Assessing the Complex Interaction and Variations in Human Performance Using Nonmetrical Scaling Methods. 2015;579-598.
- Banks VA, Stanton NA and Harvey C. Sub-systems on the road to vehicle automation: Hands and feet free but not 'mind' free driving. 2014:62;505-514.
- Fereidunian A, Lesani H, Zamani MA, Kolarijani MAS, Hassanpour N and Mansouri SS. A complex adaptive system of systems approach to Human-Automation Interaction in smart grid. 2015,888.
- Pizziola S, Tessiera C and Dehaisb F. Petri net-based modelling of human–automation conflicts in aviation. 2014;57(3):319-31.
- Mercer J, Homola J, Cabrall C, Martin L, Morey S, Gomez A, et al. Human-automation cooperation for separation assurance in future NextGen environments. 2014.
- Wickens CD, Hollands JG, Banbury S and Parasuraman R, Designing for Effective Human-Automation Interaction. 2016.
- Xu A and Dudek G, OPTIMo: Online Probabilistic Trust Inference Model for Asymmetric Human-Robot Collaborations. 2015;221-228.
- Sheridan TB and Parasuraman R. Human-Automation Interaction. 2005:1;89-129.
- Sheridan TB. Human centred automation: oxymoron or common sense?. 1995:1.
- Bustamante EA, Madhavan P, Wickens CD, Parasuraman R, Manzey D, Bahner-Heyne JE, et al. Current Concepts and Trends in Human-Automation Interaction. 2009.
- Lee JD. Review of a pivotal Human Factors Article: "Human and Automation: Use, Misuse, Disuse, Abuse". 2008 Jun;50(3):404-10.
- Parasuraman R and Riley V. Human and Automation: Use, Misuse, Disuse, Abuse. 1997: 39(2);230-253.
- Merritt SM and Ilgen DR. Not All Trust is Created Equal: Dispositional and History-Based Trust in Human Automation Interaction. 2008 Apr;50(2):194-210.
- Sarter NB, Woods DD and Billings CE. Automation Surprises. 1997.
- Prewett MS, Johnson RC, Saboe KN, Elliott LR and Coovert MD. Managing workload in human–robot interaction: A review of empirical studies. 2010:26(5);840-856.
- Ropohl G. Society for Philisophy and Technology. 1999:4(3).
- Hollnagel E. Cognitive ergonomics: it's all in the mind. 1997:40;1170-1182.
- Fujimoto T, Jürgens U and Shimokawa K. Transforming Automobile Assembly: Experience in Automation and Work Organization. 1997;1-16.
- Fast-Berglund , Fassberg T, Hellman F, Davidsson A and Stahre J. Relations between complexity, quality and cognitive automation in mixed-model assembly. 2013:32(3);449-455.
- Brolin A, Thorvald P, Högberg D and Case K. Using mobile information sources to increase productivity and quality. 2010; 450–459.
- Fässberg T, Fasth and Stahre J. Classification of carrier and content of information. 2012.
- Backstrand G, Brolin A, Högberg D and Case K. Supporting Attention in Manual Assembly and its Influence on Quality. 2010.
- Bäckstrand G, Thorvald P, De Vin L, Hogberg D and Case K. The impact of information presentation on work environment and product quality: a case study. 2008.
- Brolin A, Backstrand G, Hogberg D and Case K. Inadequately Designed Information and its Effects on the Cognitive Workload. 2011.
- Thorvald P. Presenting Information in Manual Assembly. 2011.
- Fast-Berglund Å and Blom E. Evaluating ICT-Tools for Knowledge Sharing and Assembly Support. 2014.
- Xu LD, He W and Li S. Internet of Things in Industries: A Survey. 2014:10(4);2233-2243.
- Michalosa G, Makrisa S, Tsarouchia P, Guaschb T, Kontovrakisa D and Chryssolouris G. Design considerations for safe human-robot collaborative workplaces. 2015:37;248-253.
- Wang XV, Mohammed A and Wang L. Cloud-based Robotic System: Architecture Framework and Deployment Models. 2015.
- Miller CA and Parasuman R. Beyond Levels of Automation: An Architechture for More Flexible Human-Automation Collaboration. 2003:47(1);182-186.
- Cuevas HM, Fiore SM, Caldwell BS and Strater L. Augmenting Team Cognition in Human-Automation Teams Performing in Complex Operational Environments. 2007:78(5 Suppl);63-70.
- Romero D, Noran O, Stahre J, Bernus P and Fast-Berglund. Towards a Human-Centred Reference Architecture for Next Generation Balanced Automation Systems: Human-Automation Symbiosis. 2015;556-566.
- Wolfswinkel JF, Furtmueller E and Wilderom CP. Using grounded theory as a method for rigorously reviewing literature. 2013:22(1);45-55.
- Mattsson S, Bligård LO, Fast-Berglund and Stahre J. Using Usability to Measure Interaction in Final Assembly. 2013:46(15);64-69.
- Mattsson S, Fasth and Stahre J. Describing Human-Automation Interaction in production. 2012.
- Mattsson S, Karlsson M and Fasth. Perceived production complexity – understanding more than parts of a system. 2016: 54(20);6008-6016.
- Bunge M. A general black box theory. 1963:30(4);346-358.
- Abrahamsson L, Akselsson R, Albin M, Bohgard M, Eklund J, Ericson M, et al. Work and Technology on Human Terms. 2009.
- Nersessian NJ. Opening the black box: cognitive science and history of science. 1995:10;194-211.
- Katz JA. A psychosocial cognitive model of employment status choice. 1992:17;29-37.
- Mattsson S, and Fast-Berglund. How to support intuition in complex assembly?. 2016:50;624-628.
- Mattsson S, Fasth , Dencker K, Gullander P, Stahre J, Karlsson M, et al. Validation of the complexity index method at three manufacturing companies. 2013.
- Fasth-Berglund Å and Stahre J. Cognitive automation strategy for reconfigurable and sustainable assembly systems. 2013:33(3);294-303.
- Trist E. The evolution of socio-technical systems: A conceptual framework and an action research program. 1981.
- Hendrick H and Kleiner B. Macroergonomics: an Introduction to Work System Design. 2001.
- Trist EL and Bamforth KW. Some Social and Psychological Consequences of the Longwall Method of Coal getting. 1951:4;35.
- Taylor FW. Principles of Scientific Management. 1911.
- Helldin T. Transparency for Future Semi-Automated Systems. 2014.
- Mattsson S, Partini J and Fast-Berglund. Evaluating Four Devices that Present Operator Emotions in Real-time. 2016:50;524-528.
- Mattsson S, Fast-Berglund and Stahre J. Managing Production Complexity by Supporting Cognitive Processes in Final Assembly. 2014;16-18.






