Author: IRPA AI Analyst, Chris Surdak

Last week, I was chatting with a professor friend of mine where the topic of AI and education inevitably came up. He had said that after two years of receiving repetitive, antiseptic and often flat-out incorrect writings from his students, he stopped giving writing assignments as homework. His course work had run headlong into the world of ChatGPT, and the “Grey Goo” was taking over his class.  To actually give lessons that would stick, he’d have to change things up.

This year, while he still allows the use of AI for homework assignments, all tests are given by hand, in class, in the old blue books that many of us used way back in the 20th Century. He noted with concern that many of his students had difficulty expressing themselves effectively in their handwritten papers, even when they otherwise showed mastery of the topic.

I decided that this would be a good topic for a paper and sat down to draw up an outline. Taking a short break, I launched YouTube in a window and noted a YouTube video by Upper Echelon, an edutainment channel with nearly 750,000 followers.  While I had never once viewed a video on this channel, this one was right on the money for what I was working on. The video, entitled Idiocracy is Becoming Real discusses the 2006 comedy movie where the world of the distant future is populated by not-so-smart people, after centuries of over-reliance on technology, mass media and the feeding of humanity’s more base instincts.

I’ll set aside the creepy factor of how Big Tech knew the topic of my next article about 12 seconds after I developed it, and will instead focus on how apropos this video was for my needs. Skipping past the reference to one of my favorite movies towards the more educational aspects of the video, the presenter advanced the thesis that ever-improving technology, and AI in particular puts humans at risk of becoming, well, dumber.

The video discussed several research papers which studied this effect, including:

A long time ago, I kept a road atlas under the seat of my car.  Most people did.  Today, how many of us could make it 2-3 blocks in an unfamiliar city without the use of some GPS device?

This study found that the more students relied upon automated spell checkers the worse they were at identifying and correcting spelling errors without such assistance.

This effect was even greater for students learning a new language, and indeed automation significantly impeded progress in rates of learning.

And finally, to the crux of this paper, researchers have noted that students who use generative AI to simplify or even complete their homework suffered notable decreases in cognitive skills such as critical thinking and reasoning.

Rushing Forward to Fall Behind?

While the entire world seems to be rushing headlong into advancing and adopting AI, it seems that only a few people are analyzing the consequences of this decision.  There is certainly a great deal of concern over how many people may be displaced by AI, but what about the effects AI use will have on those who remain in the workforce?  Are we inevitably heading towards an ‘Idiocracy’, or are there means by which we can protect ourselves from technology-induced stupefaction?

This should be of particular concern in professions that are deeply dependent on cognitive abilities and also demand the conjoined-yet-opposite twins of accuracy and precision. In professions such as Medicine and Law humans spend nearly a decade learning not just facts and figures but also tradecraft.  They are placed under extreme degrees of cognitive load, while at the same time accepting that the costs of being wrong can mean a client loses their case, or a patient loses their life.

Whoopsies on the Record

For these professionals, is the potential value presented by artificial intelligence worth the potential risks of dulled senses and attention to detail? In the legal world, there has been a small explosion of cases where attorneys have supplemented their work with generative AI solutions, only to find that the technology failed to meet their own duty of due care.

Examples of this include:

The legal profession is particularly suited to disruption by AI as it is extremely detail-oriented and constantly evolving.  In my own twenty years in the LegalTech industry I have direct, personal experience of how changing a single word can change the ultimate impact of a law, regulation or ruling in instance where BOTH precision AND accuracy are required.  Further, the principle of precedence means that the law is changing every single day.  It is not enough to know case law from a hundred years ago, a good attorney must also keep up to date with an unending flood of new rulings coming out every day.

Lawyerly Use of AI

Technology is particularly good at digesting and organizing vast quantities of data, something that attorneys have been taking advantage of for many decades. However, with AI attorneys are now weighing the use of technology to reason, assess and argue. While AI continues to make advances in these areas, the examples provided above should bring some pause to anyone considering the adoption of C3PO as their attorney in the near future.

There are some key principles that attorneys should follow in their use of AI for the foreseeable future:

  • Trust But Verify:  In the immortal words of President Reagan, it is OK to trust these agents to perform some of the heavy lifting of case law, but one should always double check their work.  As the attorneys in the articles above learned the hard way, the last person you want to discover your AI’s hallucinations is your judge, opposing counsel or client.
  • Less Is More: If there’s one thing that “generative AI” is good at, it’s generation of content.  It’s not unusual for a bot to say in paragraphs what could have been said in a sentence or two.  While many attorneys are paid by the hour, few are paid by the word, and in legal documents brevity is always appreciated; if only because it’s so rare. Getting concise (AKA, short) responses from generative AI usually comes down to effective prompting, and larger law firms and legal departments may benefit greatly from hiring people who are (verifiably) skilled in prompt engineering.
  • AI as Partner: If your organization does invest in a prompt engineer or other AI expert, make sure that the results of their work accumulate to the benefit of the whole firm or department.  As with other trade craft, best practices for leveraging AI should be managed and shared with everyone. This will help level-up the skills of the whole team and ensure a degree of consistency in results. With increasing use, your AI should become more skilled, more accurate and more productive, just like a human partner in a firm.
  • Creative Billing: Many legal firms and departments are facing every-growing billing pressures.  Clients balk at ever-growing fees, especially in an era where AI tools are supposed to provide break-through improvements in productivity for knowledge workers.   While AI may save you vast amounts of work, your clients will expect at least some of those savings to show up in their bills.  Attorneys will find themselves under even greater pressure to control or reduce costs, while simultaneously having to absorb the cost of adopting these new tools.  New billing models, predicated upon value-added rather than time-spent, may be required to thrive in this new environment.
  • Career Progression: Managing the progression of attorneys through their career is going to be an increasing challenge in the age of AI.  The technology can easily automate much of the grunt-work that was previously relegated to new, inexperienced attorneys.  Indeed, performing this grunt-work is how new associates learned their trade craft.  If AI can replace the work used to help train new attorneys how will new attorneys ever get trained? If AI’s require expert supervision, how will we create new experts if no new people are brought into the industry because the low-level work has been automated?

I am convinced that this last issue may grow increasingly relevant as AI enters the workforce.  As discussed in this  Thomson Reuters' article, the legal profession has already experienced such a “brain drain” in recent history.  During the pandemic, new associates were often required to work from home, missing out on a great deal of in-person interactions with senior members of their team.  After several years of remote work, many of these attorneys have suffered a permanent loss of on-the-job training that they would have otherwise received.  As firms increase their use of AI, this dissociation might grow over time.

Failure is Not an Option

I started my career building spacecraft and most of my colleagues from that time are fans of the movie “Apollo 13.” After a life-threatening accident cripples their spacecraft, the astronauts of Apollo 13 were dependent upon hundreds of ground-based engineers to come up with a way for them to get back to Earth alive.  In this true story of creativity under fire, flight controller Gene Cranz famously told his team, “Failure is not an option.”

In a world rushing head long into adoption of AI I am reminded of this credo from my spacecraft days.  While most of the engineers focused upon achieving their mission, some of us instead focused on not blowing up along the way.  Whether in space exploration or in the use of digital lawyers, this remains good advice to this day.

Particularly with generative AI, governance is critical mostly because generative AI is ungovernable. As such, effort and intent count as much with the use of AI as it does in practicing law.

About the Author: Chris Surdak 


Chris Surdak is a Senior IRPA AI Advisor and was formerly White House Chief Transformation officer, Automation & AI Practice Lead at EY & Executive Partner for Digital Transformation at Gartner. He’s an engineer, futurist, transformation executive and best-selling author, with over 30 years’ experience in technology development and deployment, digital transformation, blockchain, data and analytics and AI & intelligent automation


CLICK HERE TO SCHEDULE AN ANALYST CHAT

Links:

Originally posted on 2025-02-20 in the IRPA AI Network — Enterprise AI

Keep Reading

View more
caret-right