reuters.com
www.reuters.com

Well-known AI chatbots can be configured to routinely answer health queries with false information that appears authoritative, complete with fake citations from real medical journals, Australian researchers have found.

Without better internal safeguards, widely used AI tools can be easily deployed to churn out dangerous health misinformation at high volumes, they warned in the Annals of Internal Medicine.

“If a technology is vulnerable to misuse, malicious actors will inevitably attempt to exploit it - whether for financial gain or to cause harm,” said senior study author Ashley Hopkins of Flinders University College of Medicine and Public Health in Adelaide.

  • venusaur@lemmy.worldEnglish
    2·
    2 days ago

    The checker models aren’t trying to give you a correct answer with confidence. Their purpose is to find an incorrect answer. They’ll both do their task with confidence.

      • venusaur@lemmy.worldEnglish
        2·
        1 day ago

        Sure but they’re doing opposite tasks. You’re absolutely right that they could be wrong sometimes. So are people. Over time it gets better, especially with more regulation and smarter models.

        • vrighter@discuss.tchncs.deEnglish
          1·
          20 hours ago

          opposite or not, they are both tasks that the fixed-matrix-multiplications can utterly fail at. It’s not a regulation thing. It’s a math thing: this cannot possibly work.

          If you could get the checker to be correct all of the time, then you could just do that on the model it’s “checking” because it is literally the same thing, with the same failure modes, and the same lack of any real authority in anything it spits

          • venusaur@lemmy.worldEnglish
            2·
            14 hours ago

            That’s not how it works though. It would be great if these AI models were deterministic but you can get different answers to the same questions at any given time. Given different input and given different goals, the agents wouldn’t likely fail on the same task when given proper instruction.

            The main point is that it’s not going to be correct all the time. And neither is a human.

            The regulation comes in when you’re dealing with sensitive information, like health diagnoses. There needs to be some logic in place to stop the models from being so confident with wrong answers that could hurt people.

            Realistically, neither of us know what’s gonna work until we try it. Theoretically, verification agents would work.