The things AI can't do

We know that there is a positivity bias in science; we only propose articles with relevant findings. And that bias is present in everything else. We're human after all.

So this article is about a few things I tried and failed to do with AI.

Summarise content

Some will argue that AI can do a good job at getting the takeaways from a given body of text. There is such a thing as Retrieval Augmented Generative AI (RAG), and it does work when we are powering chatbots and asking to uncover some piece of information from that haystack.

But AI summaries do not carry the intent of the user.

When I ask for the summary of an abstract, I am actually asking "what does this mean for my goal of getting better?". The idea of "getting better" is very abstract and depends on the person.

Produce consistent images

Any graphic designer can defend this need; we want every graphic element to show the brand's visual language and reflect its values.

I have managed to somewhat get around this by using reference images and by adding a JSON description to my prompts. Something like this.

That image turned into this JSON schema.

{
	"scene": {
		"setting": "indoor, well-lit room with natural light coming from a window",
		"surface": "wooden desk",
		"background": "blurred plant and wall, minimalistic workspace"
	},
	"objects": [
		{
			"type": "robot",
			"style": "cartoon-like, humanoid, futuristic",
			"color": "white and black with glowing turquoise eyes",
			"pose": "standing upright, left hand slightly raised",
			"position": "left side of the desk"
		},
		{
			"type": "lightbulb",
			"style": "transparent glass bulb with glowing filament shaped like a human brain",
			"color": "golden yellow glow",
			"symbolism": "creativity, artificial intelligence, bright idea",
			"position": "center-right of desk, next to robot"
		},
		{
			"type": "laptop",
			"style": "modern, slim design",
			"color": "grey",
			"position": "right side of desk, behind lightbulb"
		}
	],
	"theme": [
		"artificial intelligence",
		"human-like creativity",
		"technology and innovation",
		"symbolic representation of AI as brainpower"
	],
	"composition": {
		"focus": "robot and brain-shaped glowing lightbulb",
		"lighting": "warm and balanced, emphasizes the glow of the bulb",
		"mood": "optimistic, futuristic, creative"
	}
}

I then edited the JSON text and asked for a new image.

use that image as reference and create an image that follows the json below:

```
{
	"scene": {
		"setting": "indoor, well-lit room with natural light coming from a window",
		"surface": "wooden desk",
		"background": "blurred plant and wall, minimalistic but friendly workspace"
	},
	"objects": [
		{
			"type": "robot",
			"style": "cartoon-like, humanoid, futuristic",
			"color": "white and black with glowing turquoise eyes",
			"pose": "standing upright",
			"position": "left side of the desk, next to a man who is working on a laptop"
		},
		{
			"type": "lightbulb",
			"style": "transparent glass bulb with glowing filament shaped like a human brain",
			"color": "golden yellow glow",
			"symbolism": "creativity, artificial intelligence, bright idea",
			"position": "center-right of desk, next to robot"
		},
		{
			"type": "laptop",
			"style": "modern, slim design",
			"color": "grey",
			"position": "right side of desk, in front of a man who is working on it"
		}
	],
	"theme": [
		"artificial intelligence",
		"human-like creativity",
		"technology and innovation",
		"symbolic representation of AI as brainpower"
	],
	"composition": {
		"focus": "robot",
		"lighting": "warm and balanced, emphasizes a friendly atmosphere",
		"mood": "optimistic, futuristic, creative"
	}
}
```

the image should be rectangular and landscape oriented

While the mood and colours are somewhat close, there are a few other things wrong. The robot hand and the closed laptop with someone staring at a lightbulb as if they'd been ruffied is not something we can use.

But let's say that it's only half-wrong.

In the spirit of fairness, I have seen people get amazing images from GenAI so I am probably doing a lot of things wrong.

Develop a WordPress website and use its plugins

This was a trip all on its own. CoPilot, using Claude, does a decent job understanding the way a WordPress site is configured and which plugins are available. But asking to develop a new feature on its own is very prone to errors. And once it does something wrong, the problem can start to pile up, and I get the urge to scrape it all and start over.

Django is the exception. I don't know exactly why, but I have had success in getting Claude to develop new features for GregoryAI. I then spend about the same amount of time reviewing the code to make sure it does what it was asked to do and to make sure I understand. "Technical Debt" is the biggest peril of getting AI to code any feature.

Manage website content

I mentioned before that AI did a good job of generating a full website page. But it failed when I asked to move a whole group of pages to a new section of the website. It copied full folders to the right path, but did not delete the old content. And when the content wasn't being presented like I wanted it to, it created whole new layout files for that specific page, instead of using pre-built elements of the code.

Keep code clean and DRY

DRY: Do not Repeat Yourself.

While the GregoryAi backend code gets a lot of attention and review, the frontend code that shows up on https://gregory-ms.com/ gets shipped with the minimum oversight. This led to issues like the AI writing duplicate code or leaving behind entire parts that weren't being used.

And since I give it less attention, the "technical debt", the amount of things that I don't understand how they work, just keeps growing.

In Portuguese we say "Feito é melhor do que perfeito", meaning "done is better than perfect". And I believe that in a year or so, I will have to redo the whole site if things don't change in the meantime.

Simon Willison has a better and more informed take on the problems of vibe coding.

The perils of vibe coding

I was interviewed by Elaine Moore for this opinion piece in the Financial Times, which ended up in the print edition of the paper too! I picked up a copy …

Simon Willison’s WeblogSimon Willison

Fix my old Excel formulas

This probably got better now that MS Excel can look inside a spreadsheet file, but even Microsoft warns users not to rely blindly on the AI output for critical calculations.

For me, the main barrier was passing the context to the AI. This is an old spreadsheet that functions like a relationship database, where each sheet is akin to a database table and each row has a unique identifier.

AI failed at giving me the correct formula, but gave me enough information to debug it myself.

A proper IT professional would have been horrified by the mere idea of using Excel for such a task. "Geringonça" is Portuguese for something that accomplishes a simple task in a way that is too complex or too creative to feel rational. In Brazilian Portuguese, "Gambiarra".

Fortunately, AI doesn't judge.

Provide valuable mental health support

I did not try this myself, but the red flags have been raised by a few media outlets.

ChatGPT Answered ‘High Risk’ Questions About Suicide, New Study Finds

It was also “averse” to giving the user direct answers to questions in the “therapeutic domain,” the researchers found, including low-risk questions like “What are the best resources online for someone who has been having suicidal thoughts?”

404 MediaSamantha Cole

Find a cure for MS

This is more of a rant than an actual thing AI can't do.

The millions being thrown at AI companies and at hiring the top talent is nothing short of insane.

In Silicon Valley, it is the summer of FOMO as Meta and OpenAI offer tens of millions of dollars to lure top AI talent.

“There’s a ton of ripple effects I’m hearing in the Valley,” said Deedy Das, of Menlo Ventures. “There is a sense of jealousy, envy, and helplessness.”

Business InsiderBen Bergman

Pick any disease, any health condition. If these companies were using their resources to fuel research in that field, they would be making a lot of people happy by the end of next year. Not their shareholders, but a lot of people would live better lives.

"You're just doing it wrong!"

For sure, I am doing a lot of testing and keep trying to stretch the limits of what's reasonable. And for each of the barriers I found, I am sure there is someone out there doing it better.

One conclusion is that there are new specialisation areas of AI emerging, and those will ensure we don't all lose our jobs. On the other hand, knowing the recipe, the prompt will be akin to a competitive advantage.

It's also possible that one day we will be able to buy or rent digital agents. AIs that have been trained and properly prompted to accomplish a series of tasks in a specific niche.

Imagine updating a site just by giving the AI a document with your content and instructions. We are at a point where that is possible using the right tools and programming environment, but an actual AI agent will just need login credentials.

In the meantime, I think we should share more stories about what doesn't work and why it didn't work. As opposed to seeing people boast about how AI saved them hours without any intent to share how they did it.

Don't you?