A better AI system?


Via Kevin Drum, I learned that the Wall Street Journal had tested five AI engines and the winner was one that I had not heard of before called Perplexity, where the basic version is free.

As readers may recall, back in December I tried out ChatGPT and Bard with the prompt “What can you tell me about Mano Singham?” and got back some accurate information mixed in with false ones. I read that turning out some false information was common in these AI systems, making them of highly dubious value.

I tried the prompt “Who is Mano Singham” on Perplexity and got results that were fairly extensive and correct and did not have any false information. I used a prompt asking about myself not because of vanity (at least not entirely) but because that way I can know what is true and false immediately without having to do any further research.

Here is what I got back.

Mano Singham is a theoretical physicist and educator who has worked at Case Western Reserve University in Cleveland, Ohio, USA. Here are some key details about him:

  • He obtained his BS from the University of Colombo in Sri Lanka, and MS and PhD degrees in theoretical nuclear physics from the University of Pittsburgh.
  • He served as the Director of the University Center for Innovation in Teaching and Education (UCITE) at Case Western Reserve University, where he worked to improve teaching methods and mentored faculty.
  • He is an Adjunct Associate Professor of Physics at Case Western and a Fellow of the American Physical Society. In 2001, he won the university’s Carl F. Wittke award for distinguished undergraduate teaching.
  • His research interests include education, theories of knowledge, physics, and the relationship between science and religion.
  • He has written several books, including “Quest for Truth: Scientific Progress and Religious Beliefs” (2000), “The Achievement Gap in US Education: Canaries in the Mine” (2005), and “God vs. Darwin: The War Between Evolution and Creationism in the Classroom” (2009).
  • He is an outspoken atheist and social commentator who has given talks and written articles examining topics like the achievement gap in science/math education, active learning methods, and conflicts between science and religion.

In summary, Mano Singham is a Sri Lankan-American physicist who became deeply involved in science education, teaching methods, and examining the intersection of science and religion through his work at Case Western Reserve University.

This was all correct.

In addition, Perplexity provides other result subheadings that you can click on for more information. For example, here is the response to one of the subheadings which was “how does Mano Singham combine physics and philosophy in his work?”

Mano Singham combines physics and philosophy in his work by using the tools and methods of science, particularly physics and biology, to develop a coherent personal philosophy for understanding the world. Here are some key ways he integrates these disciplines:

He approaches philosophical questions from the perspective of a physicist and scientist. As an adjunct professor of physics, he has a deep grounding in the systematic study of physics and applies its principles to examine broader philosophical issues.

His books like “Quest for Truth: Scientific Progress and Religious Beliefs” and “The Great Paradox of Science” explore the intersection of science and philosophy. They analyze how scientific theories and knowledge shape our understanding of the world from a philosophical standpoint.

While not formally trained in philosophy, he sees himself as a “natural philosopher” in the classical sense – using science as a foundation to make sense of the world and develop a rational worldview.

Specific areas where he combines physics and philosophy include theories of knowledge, the philosophy of science, the relationship between science and religion, and the role of subjectivity in education.

By drawing upon his expertise in theoretical physics as well as self-study of biology and evolution, he aims to create a coherent philosophical framework grounded in scientific principles and empiricism.

So in essence, Singham leverages his training as a physicist and deep knowledge of science to philosophically examine profound questions about knowledge, truth, human nature and belief systems from a rational, evidence-based perspective.

I can’t quibble with any of that. It all seems much better than the other AI engines I hd tried. Of course they may have gotten better over time so I tried ChatGPT again and got just one short paragraph that, while free of falsehoods, did not have much detail. Here is the response in its entirety.

Mano Singham is a theoretical physicist, a retired physics professor, and a prolific writer known for his blog “The Singham Scenario” where he covers a wide range of topics including science, politics, and religion. Singham is also known for his progressive views and his critiques of various social and political issues, particularly within the United States. He has authored several books and is a prominent figure in the secular and skeptic communities.

And as for Bard, it was a total bust. Here is what I got.

I do not have enough information about that person to help with your request. I am a large language model, and I am able to communicate and generate human-like text in response to a wide range of prompts and questions, but my knowledge about this person is limited. Is there anything else I can do to help you with this request?

So Bard had completely forgotten about me. How fickle these AI systems are!

Both ChatGPT and Bard seemed to have regressed somewhat in that they had less information about me than before. At least from my experience, Perplexity seems to be the one to beat.

Comments

  1. OverlappingMagisteria says

    ChatGPT’s brevity could be a plus or a minus, depending on your preferences. If I just want a quick summary of who you are, I’d prefer ChatGPT’s output. Less information to parse through. Perhaps you could follow up with “Can you give me more detail about him?”

    Have you ever called your blog “The Singham Scenario”, or is that a hallucination? Googling it only returns this very blog post and something unrelated.

  2. robert79 says

    Haha, so I just tried it using my own name, I don’t have a strong internet presence.

    It first got a remote uncle with the same name as mine.

    When I specified “the mathematician” it got the school I got my PhD from correct, but my thesis advisor wrong. (Famous mathematician, same country, but off by a generation.)

  3. sonofrojblake says

    Interesting. It knows a little about me, and significantly, unlike ChatGPT, didn’t hallucinate anything.

    The improvement here seems primarily that while it may be relatively light on facts, it’s entirely lacking in wrong stuff. That’s actually very impressive. I wonder what they’ve done to achieve that?

  4. John Morales says

    The inference engine part of the AI relies on the training dataset; all that this example shows for sure is that Perplexity’s dataset has more information about Mano than the others.

    Also, “I do not have enough information about that person to help with your request.” is certainly more helpful than vague generalities or random guesses, no?

  5. Holms says

    Ugh, for all its strengths, Perplexity is picking up some of the worst recent trends in writing. “So in essence, Singham leverages his training” -- yikes. Next you know it’ll be “examines politics through a ____ lens”, and at that point it will be indistinguishable from the worst of the commentators of politics.

  6. garnetstar says

    I’m still having a hard time thinking why I need these so-called AI’s. I mean, if i wanted to look up Mano, or a “famous person” who the AI would have information about, how about Wikipedia? They usually have the bare data on a person right.

    I suppose that if a person isn’t written up in Wikipedia, an AI would get you the info, but since it won’t be very accurate, is that really an improvement? How about just googling?

    And Holms @7, I SO AGREE! The prose style of these engines is just *awful*! Anodyne, bland, never make a firm point or comes to a firm conclusion, doesn’t construct an argument, the dreadful vocabulary. It’s so inescapable that I bet it’ll become easy to identify a machine-written piece by its characteristic dreadful style.

Leave a Reply

Your email address will not be published. Required fields are marked *