In March 2023 the Future of Life Institute published an open letter, calling for a pause in developing ever more powerful AI systems. They suggested that ‘AI systems with human-competitive intelligence can pose profound risks to society and humanity …..representing a profound change in the history of life on Earth, and should be planned for and managed with commensurate care and resources’

This is not a new story, more the latest phase of a continuing history, driven by techno wizardry under the disguise of improving human experiences. Here’s an old story, further back in the history but contemporary in its topics.

As I juggled the briefcase, coat and bag of duty free between the floor, the overhead locker and the person in the next seat, I apologized, as one does, about three times.

‘Sorry, is that in your way’

‘Sorry, I’ll try to fit it under the feet….’

Like me he was besuited, annoyed at the gate wait and the tarmac delay and hungry in anticipation of the meal to come, once we were airborne.

We didn’t exchange more than frantic hand wrestling for the seat belts and shrugs at the Captain’s pessimistic log until the shivers of activity by the curtains showed some nature of food was on the way.

‘Sorry’ he said ‘but in a minute they will be a dispute with the aircrew’

From his tone I could tell the dispute was going to be not unadjacent to the seat next to me

‘I’m stuck in their computer’ he went on ‘Have been for 12 months’

‘In a few moments they’ll bring a vegetarian meal for me. I’m not vegetarian but their computer is convinced I am – input error last year. I’ve written and phoned but they can’t get it changed. So I have to take a stand, sorry about the…..’

He was interrupted by the steward

‘Your vegetarian meal, sir’

‘I’m not vegetarian’

‘Oh yes you are, Mr Houseman – this is your meal’

‘No, you’ve made a mistake’

‘No, Houseman, Seat 11 B, no mistake unless you’ve changed seats with someone else.

Vegetarian meal; it’s your privilege from your Executive package’

‘Your computer is wrong – I’ve told your offices, they’re meant to correct it’

‘You don’t want your special meal?’

‘It’s not mine and no I don’t’

It was clear from the steward’s face that this man was in error and the computer wrongly accused. Either he had changed seats or his name or his persona, or his sex – the data could not be wrong. Red faced the steward strode away carrying the orphan meal ahead of him like a sacrifice to the great databank in the sky.

‘Sorry’ my neighbour said ‘I did warn you’

And suddenly he was engulfed in conversations from all sides with fellow passengers with similar stories. Special dishes, special newspapers, cross ticket promotions, even wheelchairs all had been offered, unsolicited and unwanted but ‘specially delivered for you’ by the magic of the customer focused computer.

Tailor-made marketing gone mad, customer satisfaction objective totally reversed.

The theory is indisputable – understand the individual consumer, recognize and anticipate their needs and the reward will be sales, loyalty and warm glows all round. The secrets of success are clear – state of the art database techniques, swift response and happy customers.

The biggest problem is the customers – the trouble with people is they will keep changing. Human sort of thing, really.

As the IT techo has it – we sorted the hardware and software but have problem with the warmware !

The human psyche has a way of slipping between segmentation, psychometrics, and consumer analysis. The algorithm that assumes I’m the person I was six months ago at 1030 on that rainy day I filled in the questionnaire with a hangover will turn up some very erroneous links.

Don’t tell me how I feel !! is a common cry.

We can add ‘don’t tell me you know who I am’ !

How do you feel when you’ve made it through the ‘press- ups’ (if you ….press 1 now etc) to get to the actual human being and they ask:

‘Can I have your post code’                     

‘And the first line of your address’          

‘Ah yes Mr Bourne I’ve got your details on my screen now’

Do I feel warm and welcomed and in buying mode? No, I’m feeling rather invaded and exposed as this total stranger tells me my life story.

If you want to sell to me, I want dialogue, not a big brother who thinks he knows me.

Find out who and where I am now, not what the screen tells you about where I used to be.

The best bespoke retailers could tell all this in a 30 sec glance – less time than it takes to ‘load me onto the screen’. Human beings can tell this from a two-way phone conversation.

Of course there’s a place for the extra efficiency of E-Business.

I don’t have too many emotions to reflect when buying cinema tickets, making routine bank transactions or reordering a staple purchase – give me a quick press-up……….

But if I’ve got a complaint, or I’m a new customer or I need some reassurance on a purchase…don’t beam me up, talk to me ! 

Consumers, being human, have ways of adapting. Like so many innovations that were going to revolutionize all human life, automated business will fit in where we can see its value.

Remember the way TV was going to destroy radio, until radio fitted in your ear and went everywhere with you (and left more to the imagination); remember how newspapers were going to be destroyed by TV news until they made their sales through news about TV; remember how the phone would destroy people meeting (it’s such a great innovation, said Bell, one day every town will have one!) and now people are travelling or zooming like never before to be face to face.

AI and automated business will find their place with all the previous innovations.

For the right transactions they will save time, money and build loyalty – customize me, serve me quickly, let me get on with the rest of my life.

For the wrong transactions, they will annoy, irritate and finally alienate.

The Future of Life open letter took the risks (and the potential benefits) further :

Contemporary AI systems are now becoming human-competitive at general tasks, and we must ask ourselves: Should we let machines flood our information channels with propaganda and untruth? Should we automate away all the jobs, including the fulfilling ones? Should we develop nonhuman minds that might eventually outnumber, outsmart, obsolete and replace us? Should we risk loss of control of our civilization?

AI research and development should be refocused on making today's powerful, state-of-the-art systems more accurate, safe, interpretable, transparent, robust, aligned, trustworthy, and loyal.

Humanity can enjoy a flourishing future with AI. Having succeeded in creating powerful AI systems, we can now enjoy an "AI summer" in which we reap the rewards, engineer these systems for the clear benefit of all, and give society a chance to adapt.