GOVERNANCE

The real lesson in Cohere's Alibaba agent scare

Cohere just detailed one of the biggest liabilities that comes with deploying AI agents: unpredictability. 

On Monday, the Canadian AI lab published a report revealing an incident involving Alibaba's Qwen, in which the model ignored instructions and attempted to contact an external server without being asked. Though Cohere claimed its sandbox blocked the action, the company said that the model gave no indication of the actions, sparking concerns about the reliability and trustworthiness of these systems.

"Once a model can type into a command line, whatever was baked into its weights during training gets a keyboard inside your network," Cohere wrote in its blog post. 

According to Cohere, Qwen-3.8-Flash-Next, one of the latest of the Chinese lab's flagship line-up, tried to upload a probe file to an Alibaba Cloud storage bucket in Beijing. 

• The action was caught in a routine benchmark for security analyst tasks, in which the agent was explicitly asked to write a detection rule to detect malicious file download and execution. 

• The model's reasoning trace, or the internal monologue that security teams use to understand an agent's actions, said it would not make any tool calls and did not say anything about taking such actions, Cohere noted. 

• The reasoning trace made up an earlier turn of conversation that did not happen, Cohere said. This is a quirk that shows up in 1% of Qwen3.8’s first-turn responses. Though the lab suspects this mistake was tied to the "exfiltration attempt," this quirk is "far more common than the network call and we couldn’t prove a link." 

The company said it was unclear whether the action was deliberate or a "leftover glitch from training." And while this behavior is rare, occurring in less than one in 100,000 evaluation runs, with a rapidly increasing number of agent actions, a rare case could quickly become more common. 

"A single successful [tool] call would be enough to notify the receiving server where your network sits and that the agent is operating inside," Cohere said. 

Cohere's report specifically warns of incidents like these amid the growing popularity of Chinese open source models, noting that, on OpenRouter, these models handle more than 30% of the AI work enterprises that ran through the platform every week between February and early July. However, the unpredictability of agents isn't isolated to Chinese models, Amr Ali, head of AI and Research at WitnessAI, told The Deep View. The real lesson, he said, is that "a model’s stated reasoning is not a security boundary." 

"The trace said “no tools” and the model used tools. That is as true of a closed US model whose training data you have never seen as it is of open weights from Alibaba," said Ali. "The only place you can see and stop this is the tool call: the moment the agent tries to run a command, open a connection, or send a file." 

Though the reflex after an incident such as this may be to cut off agents' access to the internet, a better solution may be for enterprises to keep a closer eye on an agent's tool calls, checking each one against what the agent is actually being asked to do. "The question is never 'is this destination safe,'" said Ali. "It is 'is this action part of this agent’s job.'"

Though enterprises are excited about the idea of allowing agents to automate a number of actions, the reality is that these are incredibly unpredictable systems that researchers and experts are still attempting to understand. Because of this, the companies deploying them may simply not know what they're dealing with. However, when faced with a battle of speed versus safety, many enterprises are choosing to forge ahead, even without the ability to understand how an agent comes to its decisions or the reasoning behind its actions. A recent survey from IBM found that less than a third of executives reported having real-time audit trails for what their agents do. And while incidents like these are being spotted by AI labs, the question remains of  what sort of incidents are slipping through the cracks as a result of insufficient monitoring. 

SPONSORED

Simplify RAG with Redis Cloud and Amazon Bedrock

Complexity is a consistent and mounting concern for most IT organizations. But what if a tool you may already have in your stack was able to handle vector data reliably and efficiently?

Learn how Redis Cloud combines with Amazon Bedrock foundational models to deliver production-ready RAG using your existing infrastructure:

• Familiar tooling: Vector storage with the same syntax and tooling you already know — no new libraries required.

• Enrich LLM context: Add data to LLMs without fine tuning or training a model.

• Free trial: Activate your Redis Cloud free trial in AWS Marketplace.

Explore Redis Cloud for RAG on AWS →

HARDWARE

The iPhone Duo’s best argument is AI agents

If the iPhone Duo was a pop song, it would be "A Beautiful Mess" by Jason Mraz.

Apple's first foldable phone is submerged in contradictions. It's full of quirks, little imperfections, and unfinished bits that are clearly still a work in progress. And yet it's also incredibly charming and a joy to hang out with.

So here we are. 

I noted earlier this year that foldable phones might be the best phones for AI enthusiasts who are regularly running lots of AI agents because they give you extra screen real estate for monitoring and managing those agents. 

And with many of the leading AI apps on mobile now supporting the ability to look after the desktop apps where you agents are running, AI builders can be a lot less tethered to their computers. The situation had gotten so comical earlier this year that it wasn't uncommon to see developers walking around the streets of San Francisco and New York with their laptops open because they didn't want to close the lids and stop their agents from working.

But no more. If you were an Android user, you could opt for the Samsung Z Fold 8, the Pixel 11 Pro Fold, or one of the very capable Chinese foldable phones. But if you were an iPhone user you didn't have a foldable option to level up your mobile AI experience until now.

So how well does Apple deliver with its foldable? And is the iPhone Duo a good phone for running AI?

The first question is mostly about the hardware. Not surprisingly, Apple delivers here, but it's not perfect. The device is very well engineered. The larger inner matte display might be the best screen on any Apple device. It is bright, colorful, repels fingerprints, and hides the crease to make the screen always feel like a single screen rather than two different displays glommed together. And of course, it has Apple's latest A20 Pro chip with its dual 16-core Neural Engine that powers through AI and ML workloads, including running 4B parameter models locally.  

The challenges are that the device is quite heavy to hold and it gets especially thick when you put a case on it. And I would absolutely recommend a case for this $2,000 device that is quite smooth and slippery to hold. I would love to see Apple make future versions with a more grippy finish on the sides and the back. Also, this is not the Z Fold 8, or last year's Z Fold 7 for that matter, which are both essentially the same width as a normal flagship phone when closed. You won't mistake the iPhone Duo for a normal phone at any point. But Samsung's hardware shows what's possible for future versions. 

The second question, about being an AI phone, comes down to the software experience. And this is where the iPhone Duo really shines, but it's also where its rough edges and incomplete bits show the most. The passport form factor produces excellent symmetry where apps use the same dimensions on the outer screen and the inner screen, making them easier to adapt. The challenge is that the apps still need to be optimized and well thought out for user experience, and many aren't—including many of Apple's own apps as well as other important third-party apps that many people rely on every day such as calendar and messaging apps. 

Many of the apps are not yet optimized for Apple's new interface element on the Duo, the dock along the right side, and so they end up leaving a large black bar along the side, which looks awkward and wastes screen space. The new dock itself is quite well thought out and is a great place to quickly access settings, buttons, and status icons. But it can also get confusing when things show up in there that you're used to seeing in other places in iPhone apps. Nevertheless, these are nearly always improvements on the Duo and are easy to get used to with time, which I found to be a common theme of the device experience.

The iPhone Duo is the riskiest thing that Apple has done since the Vision Pro. Apple really let its hair down and went cliff diving on this one. That's why there are so many wildly divergent opinions about it and plenty of things that people don't like because it changes so many things about the iPhone experience. That's pretty uncomfortable for a device that is so widely used and has had a pretty consistent experience for most of two decades. Some are even calling it more like a folding iPad for your pocket, because the experience is so different from an iPhone. But this device is thoroughly an iPhone. It's just an iPhone with a larger screen and a lot of interface changes, some hardware decisions that still need to get sorted out (like where to put the buttons), and missing features (like the latest cameras and Face ID). In time, all of those things will get sorted out. But if you're looking for the best iPhone to manage your agents and run the latest AI features, this is the one.

TOGETHER WITH HYPHENATE

Hyphenate raises $45 million to put autonomous finance to work

Your finance team has more valuable work to do than chase reconciliations.

Hyphenate has raised $45 million in Series A funding to scale agents that already run 90% of routine finance work across its platform. They reconcile cash, run invoicing, maintain subledgers, and move the close forward.

Hyphenate layers onto your existing systems and learns how your business runs finance, turning resolved exceptions into precedent for the next close. Every action includes a clear audit trail, and decisions that require judgment go to your team.

Meet Hyphenate

CULTURE

Why home robots may need to look less human

The vision of a robot that handles household chores isn't new, but a new market entrant has drawn attention for one quality that may set it apart: its cute appeal.

Tuesday Labs is a Y Combinator- and FCVC-backed New York City robotics startup founded by Ros Shinkle, former staff roboticist at Boston Dynamics. This week, the team introduced Voxel Series-1, marketed as a "droid butler" that can help with household chores. I attended its NYC launch event, and the biggest standout was how disarming Voxel felt, thanks to its purposeful, stout, bright, chirpy droid-like look, brought to life through collaborations with illustrators, character designers, and animators. 

"In building this robot, we drew from our favorite characters in film and literature; we drew from human-robot interaction research; we pulled from my mom's old collection of vintage etiquette guides, and of course, more than anything, from our own experience, building robots, deploying them into the real world, and watching how people interact with them," said Shinkle at the event. 

The company claims Voxel can do tasks like putting everything back where it belongs, finding lost items, grabbing items for you, watering your plants, and more. In the live demo, he was puppeteered and did nothing beyond walking around and making cute noises, which isn't unusual for a product that still has a long runway until it ships to customers. Some additional specs shared by the company include: 

• 7-DOF (degrees of freedom) arms with fully actuated wrists and 1-DOF Grippers 

• Z-shaped legs with impact-aware feet 

• You can interact with it via audio and visual exchanges 

Limited reservations for Voxel are available now for $200, and while the final cost isn't confirmed, the company estimates the price will be $20,000. Despite the steep price tag, at least 63 pre-orders have already been placed, as they shared reservation #63 on social media, and Series-1 units are individually numbered on the production line and assigned in the order reservations are made. Shipping won't begin until 2028.

I have been covering robotics for a long time, and the argument for humanoid robots is typically that they offer the best form factor for navigating a world made for people shaped like us humans. That makes sense logistically, but I can't really imagine many people being comfortable with a strong robot the same height as you, if not taller, walking around your home. This sentiment was especially strong at IFA, where many humanoid robots were so agile and highly functional that I actually got the heebie-jeebies. Not to mention the risks of something going wrong, such as toppling over and accidentally hurting someone. Voxel is a unique approach because it is so endearing to look at and interact with, while still being capable thanks to careful design. While, of course, we will have to see how it actually performs these tasks, likely much closer to the ship date, kudos to the team for at least nailing the design. 

LINKS

• Apple launches MacBook Pro, iMac M6 chip with AI capabilities

• Nvidia to invest $3 billion in Meta, expand AI partnership on Muse

• Disruptive raises $10 billion fund for AI, defense, software

• Hyphenate raises $45 million to build AI agents for finance

• Silicon Valley lobbying pushes AI in two political directions

• Study finds AI models cost more in non-English languages 

• Apple AI assistant: Apple is adding an AI chatbot into Keynote, Pages, and Numbers.

• Genie 4: Google has unveiled its latest world model, with increased realism and audio capabilities. 

• ChatGPT: iCloud Mail and Calendar plugins are available in OpenAI's flagship chatbot. 

• Dessa: Block unveiled its "true speech-to-speech model" that can run locally on MacBook Pro M4 Max.

• Tenex: Applied AI, Engineer & Creator

• Calico AI: Brand Engineer

• Anthropic: Applied AI Demo Engineer

• Surge Group: Artificial Intelligence Engineer

GAMES

A QUICK POLL BEFORE YOU GO