Error of the Day & Maintaining Integrity of Algorithmic Results

If you’re into algorithms, you should absolutely subscribe to the MIT Technology Review newsletter called The Algorithm.

Earlier this month, the folks at The Algorithm asked “what is AI, exactly?” The answer is reproduced below.

The question may seem basic, but the answer is kind of complicated. In the broadest sense, AI refers to machines that can learn, reason, and act for themselves. They can make their own decisions when faced with new situations, in the same way that humans and animals can.

As it currently stands, the vast majority of the AI advancements and applications you hear about refer to a category of algorithms known as machine learning. These algorithms use statistics to find patterns in massive amounts of data. They then use those patterns to make predictions on things like what shows you might like on Netflix, what you’re saying when you speak to Alexa, or whether you have cancer based on your MRI.

Machine learning, and its subset deep learning (basically machine learning on steroids), is incredibly powerful. It is the basis of many major breakthroughs, including facial recognitionhyper-realistic photo and voice synthesis, and AlphaGo, the program that beat the best human player in the complex game of Go. But it is also just a tiny fraction of what AI could be.

The grand idea is to develop something resembling human intelligence, which is often referred to as “artificial general intelligence,” or “AGI.” Some experts believe that machine learning and deep learning will eventually get us to AGI with enough data, but most would agree there are big missing pieces and it’s still a long way off. AI may have mastered Go, but in other ways it is still much dumber than a toddler.

In that sense, AI is also aspirational, and its definition is constantly evolving. What would have been considered AI in the past may not be considered AI today. 

Because of this, the boundaries of AI can get really confusing, and the term often gets mangled to include any kind of algorithm or computer program. We can thank Silicon Valley for constantly inflating the capabilities of AI for its own convenience.

It’s good to be reminded of this definition as we contend with the latest releases of the legal research databases as the databases continuously tweak their underlying algorithms — the latest being Westlaw Edge.

With Westlaw Edge comes a revised “WestSearch Plus.”

Introducing the next generation of legal search. Get superior predictive research suggestions as you start typing your legal query in the global search bar.

WestSearch Plus applies state-of-the-art AI technologies to help you quickly address legal questions for thousands of legal topics without needing to drill into a results list.

We’re starting to see a time when the Google Generation is already predisposed to not drill into a results list and now the databases are actively advocating for the users to blindly rely on the top result in the list.

Along with the consequences of fake news on algorithmic results when using Google, for example, we must also be aware of the errors within the legal research databases themselves. To that end, a fellow law librarian, Mary Matuszak, has been collecting the errors that she finds during the legal research process in the various databases and distributes them via the Law-Lib listerv as “Error of the Day.”

From October 30, 2018:

Error of the Day  A  Lexis typo (possibly scanning error) in  Excessiveness of Bail in State Cases, 7 A.L.R.6th 487.   The following group of letters is used six times throughout the document, CocainesepBail.   A quick look at the Westlaw version shows that it should be Cocaine – Bail

From November 5, 2018:

In the Case People v Kindell, 148 AD3d 456 (1st Dept 2017), Susan Axelrod is listed as both the counsel for the Appellant and the Respondent.   The official version, the print, does not list the attorneys.

[]

I confirmed with ADA Axelrod that she did not represent the defendant and opposing counsel was not someone with the same name.   I also checked the defendant’s brief and it lists Ms. Moser as counsel.

While these errors are seemingly minute individually, the consequences are greater in the aggregate.

My own mentor, a law librarian who had been in the profession for 40 years, kept a print file of the errors that he found in the databases while performing legal research. The file was overflowing by the time I saw it roughly 3 years before his retirement.

Because an algorithm’s results are only as good as the underlying data, as we move toward an algorithmic society that relies heavily on algorithmic decision making, these errors could have consequences on the development of the law.

The Duty of Tech Competence & AI

The use of artificial intelligence has many potential pitfalls regarding attorney professional responsibility rules. One such pitfall concerns the duty of technology competence.

imagesAs Robert Ambrogi points out over on Law Sites, a majority of states have now adopted the duty of technology competence for lawyers – first noted in Comment 8 to ABA Model Rule 1.1.

The ABA version states:
To maintain the requisite knowledge and skill, a lawyer should keep abreast of changes in the law and its practice, including the benefits and risks associated with relevant technology, engage in continuing study and education and comply with all continuing legal education requirements to which the lawyer is subject. (Emphasis added.)

While the states may differ in the exact language of their rules, these rules will likely have an ongoing effect on a lawyer’s duty to learn various aspects of ever-changing technology.

Down the road, there may be a time when attorneys must know and understand how artificial intelligence works to be able to rely on technology to perform the more sophisticated functions of law practice.

As lawyers begin to use ROSS, say, to perform legal research or even draft simple memos, it is not unreasonable to presume that a lawyer would need to understand how ROSS decided a particular issue to have true algorithmic accountability. Because a technology like ROSS cannot be subject to the same professional responsibility rules as a living, breathing lawyer, it is up to the lawyer to maintain a duty of technological competence to understand and vet the work of the software.

This is tricky because we are currently at a point where most algorithms are proprietary and there is little transparency about the results that are generated. It is unlikely that this competing issue with be resolved anytime soon.

Until such time when the AI developers release the very decision trees for how an algorithm came to a particular result, law librarians will be helpful in teaching lawyers to understand the current state of AI technology. During our legal research instruction, we should offer pointers to lawyers on the results generated and how to spot possible issues, such as bias.

Bloomberg Law: Litigation Analytics – What is it?

This is the first of a four part series spotlighting Bloomberg Law’s Litigation Analytics.

Bloomberg Law’s Litigation Analytics is a newly developed tool that allows a user to search current litigation statistics by firm, by represented client, or by judge. It contains dynamic charts for the user to interact with.

To access this feature, you will need your Bloomberg Law login. After logging in, you will click on the litigation and dockets tab at the top of the page. Under Litigation and Dockets, select Litigation Analytics.

This is the starting homepage, displayed is an example of search by company, specifically Apple Inc.

BLA -1

This is an excellent tool to research all kinds of litigation, such as what firms are currently representing some of your clients in another jurisdiction, which firm represents an opposing party, the frequency of appearances for each firm, and what judges are hearing these cases. Data is downloadable into PDF, Word, and Excel Reports. After you locate the data that you need, click on the printer icon on the right hand side, then this window will pop up with a variety of options.

BLA -2\

To begin, you will want to select what type of party you are searching at the top. This will either be company, law firm or judge.

BLA -3

For more general information about the resource, visit https://www.bna.com/litigation-analytics/.

Access to the Bloomberg Law database is available through the Texas Tech Law Library website under the Electronic Databases tab.

IBM Watson For Legal Research Coming Soon

IBM’s Watson is close to becoming realized in the legal research realm.

According to The Globe and Mail, a class project-turned-startup launched by University of Toronto students that uses IBM’s artificially intelligent Watson computer to do legal research now has backing from Dentons, the world’s largest law firm. Called Ross, the app uses Watson to scour millions of pages of case law and other legal documents in seconds and answer legal questions. Its founders liken it to a smarter version of iPhone’s Siri, but for lawyers, and say it could one day replace some of the grunt research work now done by low-level associates at the world’s top law firms. It is one of several attempts to apply what is called “cognitive computing” to the historically technology-averse legal profession.

And Ross is learning quickly. One of Ross’s developers noted: “It’s early days for sure.” “But what we are seeing is Ross grasping and understanding legal concepts and learning based on the questions and also getting user feedback. … Just like a human, it’s getting its experience in a law firm and being able to learn and get better.”

This will eventually have major ramifications for legal research as we know it. As mentioned in the article, this will likely replace much of the grunt research like finding particular statutes or cases by citation. But Ross is nowhere near being able to creatively use case law to form arguments. And there are many issues to be worked out with Ross storing proprietary information.

While there is no denying that Ross will help augment intelligence, he should be considered more of another tool in a lawyer’s toolbox rather than a replacement. Think of Iron Man’s JARVIS as opposed to The Terminator.

Jarvis_shield_interface