What are artificial intelligence and algorithmic management systems ?

4 min read

Is your boss a computer program? Unless you work for a gig economy platform like Deliveroo or Uber, probably not. But behind the scenes, artificial intelligence (AI) and algorithmic management systems (AAMS) are proliferating in the Eu­ropean services sector -- affecting many ‘ordinary’ workplaces such as offices, banks, restaurants, and contact cen­tres. These AAMS are not typically replacing traditional managers. Rather, they are being embed­ded in the enterprise software which most firms already use to organise work and business processes. This means that the average worker is increasingly subject to the control of 'bossware' -- even if they don't yet know it.

A recent report I authored for FES and UNI provides an overview of the surprising range of AAMS functions, which can increasingly impact virtually all aspects of work. By harnessing huge volumes of data on employees and their work, AAMS give bosses powerful new tools to 'coordinate, direct, evaluate and discipline' workers. AAMS augment managerial tasks throughout the employee lifecycle, from recruitment and onboarding through training and development, to task allocation and performance management. New generative AI systems like ChatGPT simply enhance the capabilities of tools already offered by AAMS vendors. These vendors loudly acclaim their advantages and benefits for third party companies. They promise to help managers increase efficiency and productivity, take better-informed decisions, identify weak points in business processes and worker performance, and improve predicitions about future performance and demand.

However, AAMS pose clear threats to work­ers. Despite occasional warnings to the contrary, the risk is not so much that they threaten the widespread replacement of workers over time (the logic of such claims seems mixed at best). Much more immediately, AAMS harbour the risks of illegitimate and expansive forms of worker surveillance, a major intensification in the pace of work, the creation of knowledge imbalances between workers and managers, and (often poor) decisions or recommendations for managers being generated automatically without sufficient oversight. While it is obvious how these systems pose a threat to workers, AAMS may also pose threats to companies which deploy them without careful consultation and consideration.

Many of these issues stem from poor understandings of what 'AI' actually is. Such systems are not in fact ‘intelligent’ at all, but simply pattern recognition tools which make their predictions and recommendations based on past correlations. When used in the workplace, it is vitally important that bosses understand the very real limitations of such technologies. Research shows how four fallacies can lead to misguided deployoment of AAMS. First, they can be given tasks which they are not actually able to perform (impossible tasks). Second,they can be badly designed and implemented (engineering failures). Third, they may hit unexpected barriers when deployed in the real world (post-deployment failures). Fourth, they may have their actual capabilities overstated or misrepresented by vendors (hype).

My report examines four specific examples of AAMS: two for sales work (Salesforce and ActivTrak), and two for warehouse management (Infor and Blue Yonder). They demonstrate how AAMS help managers exert granular control over individuals and teams, using algorithically-generated performance metrics and rec­ommendations for managerial action (up to and including firing workers). They also demonstrate the extraordinary information imbalance produced by AAMS, which leave workers in the dark about how managers evaluate their performance.

Given that protective legislation is slow to arrive, trade unions should urgently seek to engage in collective bargaining over AAMS in the European services context. This means insisting on full and transparent auditing of companies' existing AAMS systems. Furthermore, worker representatives should insist on active engagement and consultation throughout the implementation process in the future, from the purchase and rollout of new systems, to regular feedback and adjustment to ensure workers' rights and conditions are upheld throughout the lifecycle of AAMS products.

At present, AAMS pose a clear threat to workers' rights and conditions across a range of industries. If AAMS are used responsibly and transparently in the interests of both workers and firms, they can improve work and reduce repetitive tasks, leaving workers free to develop skills and focus on creative aspects of their roles. Getting there will require devotion of resources to understanding the range of new technologies that are in play, combined with strong grassroots organising and negotiating.