Challenge and complaints are now fair game (Edition 52)

A conveyancing trade piece published recently, on Legal Support Network, made a point worth noting. Practising conveyancers described a pattern spreading through their firms: clients are increasingly running their solicitor’s advice back through ChatGPT and returning with technical follow-up questions the tool generated, rather than questions arising from their own understanding. Not a complaint. Not even a challenge yet. Just a second opinion running in the background of every conversation a professional has with a client.

A lot of surveyors already know this feeling, because it happened here publicly first. Some months ago I wrote about a client here on LinkedIn who uploaded a survey report into ChatGPT and had a list of follow-up questions and challenges back within minutes. The response from this community was enormous, hundreds of reactions, dozens of comments and messages, and one joke that has stuck with me since: charging clients an extra fee if they used AI to read their own report.

Something in the relationship between surveyor and client had shifted, and nobody quite had the language for it yet.

The three stages

Stage one is the question, arriving before you have even delivered your work, sometimes before you have started it.

Stage two is the challenge, arriving once the client has your report or your advice in hand and has run it past a model that will confidently disagree with you if asked to.

Stage three is the complaint, the stage many professional bodies are talking about.

The complaints stage is where the numbers are loudest.

Greg Tsuman, a former Propertymark president, told LANDLORD TODAY

With the rise of AI, we have seen a spike in complex and intricate complaints. We are now having to spend time and effort addressing multipart points and complaints that are sometimes without grounds.

He was direct about the mechanism behind it too:

AI tends to be very much user-biased, it can tell you what you want to hear, and it can lead people into a potential false sense of awareness where the complaint might be without true merit.

Propertymark’s own guidance, published in December last year, describes these complaints as intimidating, built on templated language and sometimes on assumptions the AI got wrong, and is honest that firms may use AI to help draft a response provided “final decisions must always rest with a trained professional.” This I would certainly agree with.

The Financial Ombudsman Service has put a number on it. In an insight piece published earlier this year, it disclosed that complaint submissions are now running past two hundred pages in response to six page provisional decisions, frequently produced by claims management companies charging consumers a fee for the privilege. The regulator said it is sharing what it is finding with the FCA and the SRA, treating this as a live conduct question rather than a technology curiosity.

Then, last week, the BBC reported that schools, councils and other public bodies are under the same pressure. One council said a complaint about uncollected bins, once a matter for a single side of A4, now ran to somewhere between nineteen and twenty seven pages, citing case law and Acts of Parliament.

Deborah Evans , Chief Executive of Lawyers In Local Government and Chair of the The Pensions Ombudsman, which represents 5,600 council lawyers, told the BBC News her organisation would need double its current workforce to cope, and that AI has “increased the number of trivial or unjustified complaints.” The Information Commissioner’s Office issued guidance in May specifically for public authorities struggling with AI-generated Freedom of Information requests, citing the same rise in volume and complexity.

The Legal Ombudsman‘s latest annual report recorded 14,259 new complaints this year, a rise of 37 percent, with residential conveyancing driving almost three quarters of that increase.

The Ombudsman’s own report attributes this rise specifically to delays, poor communication and post-completion registration problems, not to AI. Separately, though, legal compliance specialists are reporting something that number does not capture. Infolegal described solicitors receiving “increasingly long, unfocused, vague and often aggressive complaints” that are “clearly generated using artificial intelligence tools,” some containing the AI’s own leftover prompts, left in the text by mistake, never meant to be seen by the firm reading them. I have no doubt a large part of increased compliants in down to AI.

And there is a cost line hiding inside all of this. CEDR, the default adjudication provider for most RICS-regulated firms, charges the firm a case fee once a complaint is accepted for adjudication, regardless of the outcome. That fee is payable even when the firm is entirely right and the complaint is dismissed. A rise in AI-inflated complaints reaching adjudication is not just a time cost.

Yale researchers analysed 1.1 million complaints filed with the US Consumer Financial Protection Bureau and found that AI-assisted submissions were measurably more persuasive and more likely to succeed.

London’s chief traffic adjudicator recorded 43,200 parking appeals last year, up almost 24 percent, with 45 percent succeeding for the motorist, numbers that would have been unthinkable for most people before a chatbot could draft a coherent appeal in ninety seconds.

AI is giving people without the money for legal advice a genuine route to challenge decisions that used to go unchallenged simply because fighting them was too expensive or too intimidating. That is a shift in who gets to hold professionals to account, and on balance, it is not a bad thing.

Spotting AI writing will become much harder

The Economist published a study on AI writing this month, testing ChatGPT, Claude, Gemini and Grok against 1.2 million words of human writing, including its own journalism, CNN, the New York Times, the Washington Post, and excerpts from novels published between 1950 and 2022. Its conclusion: the tells everyone still relies on are breaking down. Em dashes, the classic giveaway, are no longer a reliable signal across most models.

There is no single AI style, in the same way there is no single human style. A separate international safety study found that after a five minute exchange, people misidentified AI-generated text as human-written 77 percent of the time.

Can we tell when something in front of us was AI-assisted, is the wrong question

You cannot, reliably, and the gap is closing fast. The question that protects a firm is whether it can prove what it did and why, at every one of the three stages, regardless of who or what wrote the thing questioning it. The RICS AI standard has been mandatory since 9 March and does not operate retrospectively. Records not kept in spring cannot be created in September, however detailed the letter asking for them.

Free AI Briefing on this exact topic

That is what I am covering on Wednesday 16 September, a free AI Briefing session. I will cover:

– Why questions, challenges and complaints from AI-assisted clients are rising across property and the professions, and why trying to spot AI involvement is not where your time should go

– Why a proportionate response costs you less time than an exhaustive one

– Whether you should use AI to help draft your own response, and where the line sits between a useful first draft and a decision that has to stay human

– Whether your complaints process was built for this, or whether it is already out of date without anyone at your firm having noticed

– The one evidence habit that ties all of this together, so you can prove what you did, not just explain it

Free, 60 minutes. September 16th 1-2pm.

Register here in less than a minute and add to diary

Nina