UL NO. 454: The First AI Breaches

SECURITY |
AI |
PURPOSE
UNSUPERVISED LEARNING
is a newsletter about upgrading to thrive in a world full of AI.
It’s original ideas, analysis, mental models, frameworks, and tooling to
prepare you for the world that’s coming.

TOC

• SECURITY

• AI / TECH

• HUMANS

• IDEAS

• DISCOVERY

• RECOMMENDATION OF THE WEEK

• APHORISM OF THE WEEK

Hey there!

• We saw the comet yesterday! Was quite bright even to the naked eye
  between 7:15 and 7:45.

Credit: Bunny via iPhone 16 Pro

• I wrote a tutorial on how to use any Hugging Face model within Ollama!
  So now, instead of a couple of dozen models, you can use
  thousands!
  MORE

• My buddy Marcus Hutchins and I disagree about 1) whether Elon is a
  real builder who will continue to innovate and 2) whether he still has
  liberal ideals in him or if he’s permanently far-right now.
  So I offered 3 bets: 1) that Tesla stock would hit at least $250
  by June 30, 2025, and 2) hit at least $300 by December 31, 2025, and 3)
  that Elon would publicly oppose Trump on some
  liberal/authoritarian/freedom issue by December 31st, 2025.
  THE LINKEDIN THREAD

• I did a talk for the WIPO UN Group, and it went really well. Thanks
  to
  Olivia Fabreschi
  for being not just a great host but someone who’s clearly thinking about
  these things herself. Someone to watch for sure!
  OLIVIA ON LINKEDIN
  |
  THE TALK

Human 3.0—The Skills & Mental Frames Required To Thrive In An AI World

Sponsor

Your OAuth risk investigation checklist 

For most employees, OAuth grants provide a familiar “easy button” for
creating new accounts or integrations.

But, OAuth grants have also been exploited by attackers for nefarious
purposes. It’s good practice to regularly review your organization’s OAuth grants
to identify any that are overly permissive or could be malicious.

This blog post covers four key areas to look at when assessing an OAuth grant, including a free template
you can use to make sure you cover all the important steps in your OAuth
reviews.

nudgesecurity.com/post/your-oauth-risk-investigation-checklist

Get the checklist

   

SECURITY

An attacker has accessed
Muah.ai‘s AI chatbot database, exposing sensitive user interactions with AI
chatbots, including sexual fantasies. And the user accounts were linked to
peoples’ personal email addresses.
MORE

As per usual, most “AI hacking” will be normal hacking of regular
infrastructure used by AI companies.

Yes, there will be lots of input validation issues and prompt injection
and all that, but the vast majority of the damage will come from customers
giving their souls to small startups in the AI assistant / AI girlfriend
spaces.

The fundamental issue is that AI gets exponentially better the more
honest and forthcoming you are with it. Give it your trauma, your NSFW
political opinions, and your sexual fantasies, and you’re going to have a
companion that really gets you.

But when that little 9-person startup (who has no security whatsoever)
who made that bot gets hacked, all that data you gave it will be there for sharing/selling. The problem is, this won’t stop people from doing it. The tech is too
compelling. And people are too lonely.

Casio says a ransomware attack led to the theft of sensitive data,
including personal information of employees and business partners. The
attack, claimed by the Underground group, involved over 200GB of stolen
data, but credit card info was reportedly not affected. MORE

MITRE has introduced the Caldera Bounty Hunter plugin, which allows users to simulate full cyber attack chains. This tool is
designed to enhance cybersecurity training and testing by providing a more
comprehensive emulation of potential threats. MORE

Horizon3.ai
researchers detail how they identified new vulns in Palo Alto Networks’
products to achieve full system compromise.
MORE

The Internet Archive’s “The Wayback Machine” was breached, exposing lots of
user data in the 6GB SQL database of 31 million user records. The site’s
still down but they’re working to get it back up.
MORE 

Researchers from ESET have discovered two sophisticated toolsets used by a
nation-state hacking group, possibly Russian, to breach air-gapped devices.
MORE

Sponsor

Facing Alert Overload? Get the 2024 SOC Efficiency Report

Alert fatigue and analyst burnout are rising while traditional SOC tools
fall behind. Sponsored by Dropzone AI, the 2024 Osterman Report, Making the SOC More Efficient, reveals how AI-driven innovations can enhance SOC performance, reduce false positives, and cut response times. Download the report for actionable insights.

content.dropzone.ai/osterman-research-soc-survey-report

Download the Report Now

   

Cybernews says Google’s Pixel 9 Pro XL sends data packets to Google every 15
minutes, including location, email, and phone number, even with GPS off.
They claim the phone uses nearby Wi-Fi to estimate location.
MORE

The UNODC warns that Southeast Asian scammers are using deepfakes to enhance
“pig butchering” scams.
MORE

A Chinese hacking group, Salt Typhoon, has exploited back doors meant for
lawful data requests, posing a major national security risk. Verizon,
AT&T, and Lumen Technologies were among the affected companies.
MORE

Ukraine has sentenced two hackers linked to Russia’s FSB and the Armageddon
group to 15 years in absentia for cyberattacks on state institutions.
Armageddon, active since 2013, is a major state-sponsored threat actor
targeting Ukraine and its allies.
MORE

OpenAI has stopped over 20 foreign operations using its stuff to sway
political opinions and meddle in elections. Attackers used ChatGPT to create
fake articles and spearphishing campaigns.
MORE

Private intelligence firms like Recorded Future and Flashpoint are changing
intelligence by leveraging tons of data from the internet, including the
dark web, to counter global threats. I love the dynamic of startups
competing with corporations, and I love this analog of small intel shops
competing with larger state actors (in some ways).
MORE

Popular car brands like Hyundai, Kia, and Tesla are collecting driver data,
including voice recognition and camera footage, and sharing it with third
parties, according to a Choice investigation. The report found that 7 out of
10 car brands have concerning privacy policies, with Hyundai, Kia, and Tesla
being the worst.
MORE 

The Pentagon said the US will send a THAAD missile defense system to Israel
(along with about 100 US troops to operate it) to improve Israel’s defenses
against Iran.
MORE

Continue reading online to avoid the email cutoff…

AI / TECH

If you use chatGPT, try this prompt just for fun (it’s going around some
forums).

From all of our interactions together, what is one thing you can tell me
about myself that I may not know about myself.

Then after it gives you an answer, ask it for another:

Awesome. Thank you. Can you tell me something else I may not know about
myself?

Follow it up with:

Can you see any areas where I may hold myself back?

Let me know what you get back, and if you found it interesting. Honestly
mine sounded very complimentary and little like a horoscope. Kind of felt
like a scam in that way. Designed to make me feel good about myself, you
know?

Well yeah! (puffing out chest) lol

I’m skeptical of outright flattery from strangers.

Curious if any of you get something that is actually revealing vs. just
complimentary.

—

Apple’s AI researchers found that large language models (LLMs) from Meta
and OpenAI struggle with basic reasoning. They introduced a new benchmark,
GSM-Symbolic, to measure this, which found that minor changes in query
wording can lead to different answers. I find it interesting, but I’d say
that it’s easy to disrupt its reasoning rather than that it has none—which
is what a lot of the analysis is saying. MORE | THE PAPER

Geoffrey Hinton, often dubbed the godfather of AI, has won the Nobel Prize
in physics for his early work on neural networks, alongside John Hopfield.
Notably, Hinton is now firmly in the doomer camp, which is worth paying
attention to. You can’t give someone a Nobel prize and then ignore other
advice on the same topic.
MORE

Elon Musk unveiled Tesla’s new robotaxi, a self-driving electric vehicle
without a steering wheel or pedals, at the “We, Robot” event. The design
features butterfly doors and wireless charging, but it needs regulatory
approval before production.
MORE

There was so much hate against this event, and it’s revealed this love or
hate binary thing with Elon. I don’t know many people who see Elon as
complex. Nope. He’s super one thing or the other. He’s either the Saviour
of the Universe, or he’s Tech Hitler. That’s it. Pick one.

I find this highly disappointing. People seem to have lost the ability to
continue learning about someone once they’ve decided they hate or love
them. People are allergic to subtlety. They want crisp, clear answers of
Good or Evil.

This massively limits your ability to deal with the world because you’re
going to be so wrong about so many things. Reality isn’t 1 or 0 like that.
And the more subtlety you’re comfortable with, the better your probability
adjustments can be.

My read, and my prediction, on this event, is that it was a lot of vision
and hype, but that he definitely is working on the robotaxi. Will it come
out when he says? Probably not. He’s been wrong about so many timelines.

But what he’s showing is that he’s excited, and moving forward, and that
robots (Optimus) is a very real thing for him.

These events are about hope and about the existence of a man and a set of
companies that continue to try to push for the impossible. Find me anyone
like that—who can actually execute—and I guarantee you I can find a
thousand horrifically dumb things they’ve said or believed.

It comes with the territory. If you have a genius creator, you’re
unlikely to have someone who hits timelines perfectly and acts
normally.

I think that most of Elon’s innovation critics suffer from a lack of
reading enough biographies of great people. They often look a lot like
Elon. Nuanced. Complex. Broken. Genius. And flawed.

And that’s the combination that leads to them being taught in
school.

Dell’s sales staff were given just two days’ notice to return to the office
full-time, causing panic among parents struggling to arrange childcare. The
abrupt policy shift, aimed at boosting productivity, has led to crowded
offices and left some employees considering using PTO to manage family
commitments.
MORE 

Billionaire Robinhood co-founder launches Aetherflux, a space-based solar
power startup. Baiju Bhatt’s new venture aims to create a constellation of
satellites in low Earth orbit to collect and transmit solar energy using
infrared lasers. Sounds rad, but it is technically a space laser.
MORE

The US Department of Justice is considering breaking up Google after a court
said they’ve crushed competition. The DOJ accuses Google of using products
like Chrome and Android to maintain its search monopoly, leading to high ad
prices and degraded services.
MORE

Ticketmaster is the first to use Apple’s upgraded Wallet tickets for iOS 18,
giving us stuff like venue maps, parking, Apple Music playlists, and weather
forecasts. Thank god. Anything to make Ticketmaster suck less.
MORE

A new HBO documentary claims Canadian crypto expert Peter Todd is the
mysterious inventor of Bitcoin, Satoshi Nakamoto. However, Todd dismisses
the theory as “ludicrous,” stating he was too busy with school and work at
the time. Exactly what Satoshi would say…
MORE

Four Taiwanese employees at Foxconn’s Zhengzhou plant, the world’s largest
iPhone production facility, have been detained by Chinese authorities. The
detentions, likely politically motivated, come amid rising tensions between
China and Taiwan.
MORE

HUMANS

It looks like
Christopher Columbus was a Sephardic Jew from Western Europe. MORE

JPMorgan and Wells Fargo report a dip in profits. They said it was
geopolitical tension.
MORE

Your Brain Changes Based on What You Did Two Weeks Ago
MORE 

The American Heart Association outlines a strict protocol for taking blood
pressure, including sitting calmly with an empty bladder and using a bare
arm, which is frequently ignored.
MORE

Boeing is cutting 10% of its workforce—17,000 jobs—due to a tough year
marked by grounded planes, legal issues, and strikes.
MORE

Federal emergency workers in Rutherford County, NC, were temporarily
moved after reports of an “armed militia” threatening government
personnel. (see Ideas) MORE

Elizabeth Landau says single-cell cyanobacteria can anticipate seasonal
changes by sensing day length and preparing for winter. This discovery
suggests that seasonal tracking is fundamental to life, even in
short-lived organisms. MORE

United Airlines is adding new routes to lesser-known destinations like
Bilbao, Faro, Madeira, Sicily, and Nuuk, aiming to attract travelers tired
of crowded hotspots.
MORE

In his journals, Alexei Navalny, the Russian opposition leader, shares his
journey from being poisoned with Novichok to his arrest upon returning to
Russia.
MORE 

Retail sales jobs have dropped from 7.5% to 5.7% of employment over the last
decade, losing 850,000 positions despite the U.S. adding 19 million jobs
overall.
MORE

Likely due to weight loss drugs like Wegovy and Zepbound, the US adult
obesity rate has dropped by about two percentage points from 2020 to 2023.
MORE

New GLP-1 weight-loss drugs in pill form are in late-stage trials,
potentially replacing weekly injections like Wegovy and Ozempic.
MORE

Darya Kawa Mirza, a self-taught Kurdish astrophotographer, captured the
moon’s surface in stunning detail by stitching together 81,000 images into a
708-gigabyte composite.
MORE

IDEAS

Gullibility, Not Disinformation
I don’t think the US has a
misinformation problem. I think it has a gullibility problem. It’s not that
we’re being fed too much crap. It’s that we’re eating it.

Some
too-large number of Republicans now believe that Democrats are sending
hurricanes to Florida because it’s election time. That’s a population
problem. An education problem. Not a conspiracy theory problem.

In InfoSec terms, we need to reduce our vulnerability—not try to get remove
the threats. The threats will always be there. And they’ll get better.

Our only chance of fixing this is education about how the world actually
works—which both the far left and far right seem to have lost touch with.
Remember, anti-vax was a far-left thing before it was far-right. Both sides
have lost their minds.

MORE (2020)

DISCOVERY

swarm — OpenAI’s new (experimental) framework for building and
orchestrating multi-agent systems.
MORE

Command Line Tools I Like (2022) — The author shares a list of
favorite command line tools, many written in Rust, that enhance productivity
with modern features. Highlights include neovim for its Lua
scripting and LSP support, fzf for fuzzy searching,
bat for syntax-highlighted file viewing, and
exa for colorful directory listings. Other tools like
rg, fd, delta, tldr,
zoxide, and HTTPie offer improved functionality
over traditional Unix commands.
MORE

zvm — A better vim mode for zsh. LOVE this thing. Basically
highlighting and all sorts of stuff including using the Surround plugin—all
in vim mode.
MORE

Theneo 3.0 — AI-powered API documentation tool that streamlines
the creation and management of API docs.
MORE

I updated my post on Dynamic Content Generation. I think this going to be
insanely disruptive to so many industries.
MORE

Augment UI
— Use AI to prototype front-end designs. This tool helps designers quickly
create and iterate on UI concepts using artificial intelligence. MORE

Software Engineer Pay Heatmap Across the US
MORE

The Digits of Pi are Not Random
MORE

Passbook — Lets you create an Apple Wallet pass from any QR code and export
it to Wallet.
MORE

How I Animate 3Blue1Brown — A behind-the-scenes look at how 3Blue1Brown
creates its captivating math animations.
MORE

RECOMMENDATION OF THE WEEK

If you want to calm your nerves during this next month and a half, go read
about the civil rights movement and how much the country was divided then.

We’ve survived some really bad stuff. We probably will again.

APHORISM OF THE WEEK

❝  

What is to give light must endure burning.

  Victor Frankl

Become a UL Member