Starship : success or not?; Crazy story from OpenAI early days; Smart glasses show what to say on dates & more
AI agents ask each other out on dates
Hi,
This is Thomas, Cofounder and CEO of digital agency KRDS (more about me at the end, see our latest game showreel here).
You're receiving Future Weekly, my personal selection of news about some of the most exciting (and sometimes scary) developments in technology 🤖 summarized as bullet points to help you save time and anticipate the future 🔮.
First, you'll find small bites about many different news, and then further down these summaries:
AI existential risk: the doom and anti-doom arguments summarized
Starship: success or failure?
Oxford academic Nick Bostrom on why AI may be close to showing some sentience
Crazy story from the early days at OpenAI
Why OpenAI’s CEO Says the Age of Giant AI Models Is Already Over
See these AI Agents Plan Parties and Ask Each Other Out on Dates in 16-Bit Virtual Town
Small Bites
Fun prototype: new smart glasses tell you what to say on job interviews and dates using GPT-4 (see the 1-min video)
Practical text-to-video generation is here: see latest progress in text-to-video AI generators, see some examples of prompts and video outputs by NVIDIA research efforts
See also examples by Runway, another company
Decreasing lithium price boosts climate tech startups (source)
Universities should return to oral exams in the AI and ChatGPT era. (link)
Alphabet is merging its 2 AI labs Google Brain and DeepMind: welcome to Google DeepMind (source)
To be led by Demis Hassabis who was leading DeepMind
While DeepMind seems to be a bit behind OpenAI lately, let's not forget its greatest achievement so far:
DeepMind released in 2022 an AI program that created a 3D mapping of all 200 million proteins known to science. (source)
until then, it took a whole PHD, so 5 years, to do one protein structure experimentally, so doing it for all 200 millions proteins is equivalent to a billion years of PHD time
This new protein database is now public. It has been used to develop malaria vaccines, new enzymes that can eat plastic waste, and new antibiotics, said CEO Demis Hassabis.
CBS asked Alphabet's CEO Sundar Pichai about how safe it was for Google to "turn [its AI] loose on society" if its own developers "don't fully understand how it works."
The CEO's retort: "I don’t think we fully understand how a human mind works, either." (source)
In 1945, the US army conducted the Trinity test, the first detonation of a nuclear weapon. Beforehand, the question was raised as to whether the bomb might ignite the Earth’s atmosphere and extinguish life. Nuclear physics was sufficiently developed that Emil J Konopinski and others from the Manhattan Project were able to show that it was almost impossible to set the atmosphere on fire this way. But today’s very large language models are largely in a pre-scientific period. We don’t yet fully understand how they work and cannot demonstrate likely outcomes in advance.
(Financial Times, issu de la tribune "We must slow down the race to God-like AI", source, unblocked source)
PaLM-E, created by researchers at Google, uses a Large Language Model trained using sensor data as well as text, to control a robot. It can understand and carry out tasks such as “bring me the rice chips from the drawer” or “push the red blocks to the coffee cup.” (The Economist, 50-sec video, 8-min video)
Sam Altman has put almost all his liquid wealth in recent years in 2 companies. (Wall Street Journal, unblocked)
He has put $375 million into Helion Energy, which is seeking to create carbon-free energy from nuclear fusion and is close to creating “legitimate net-gain energy in a real demo,” Mr. Altman said.
He has also put $180 million into Retro, which aims to add 10 years to the human lifespan through “cellular reprogramming, plasma-inspired therapeutics and autophagy,” or the reuse of old and damaged cell parts, according to the company.
He noted how much easier these problems are, morally, than AI. “If you’re making nuclear fusion, it’s all upside. It’s just good,” he said. “If you’re making AI, it is potentially very good, potentially very terrible.”
Finding this interesting? Share it on Whatsapp, just click here ❤️
If yes, feel free to take 3 seconds to forward that newsletter to one person, or share it on Whatsapp clicking here, I'd be immensely grateful 🙂
If that email was forwarded to you, you can click here to subscribe and make sure to receive future editions in your mailbox (many CEOs and startup founders are subscribers)
More to chew!
AI existential risk: the doom and anti-doom arguments summarized (source)
The anti-doom argument is:
“We will design/engineer AGI to be safe; we will test it, monitor it, install safeguards; we will have multiple AGIs and they will police each other, etc.”
“Believing doom = believing all of these will fail = conjunction of many special arguments.”
The doom argument is:
“You can't engineer safe AGI; it will outsmart you, trick you, evade your safeguards; if you have multiple AGIs they will collude against you, etc.”
“Believing non-doom = believing you will succeed at all of these = conjunction of many special arguments.”
Each side believes that the *other* side has a weird conjunction of many dubious arguments, so each side thinks that their position is the normal default thing and that the *other* side has made an extraordinary claim that requires extraordinary evidence.
The Starship era begins
The rocket took off, reached a peak altitude just short of 40 km and then blew up
Previously, there had been 7 test flights of the Starship between July 2019 and May 2021 (without the first stage Super Heavy though), reaching a max altitude of 12km
See the 5minutes from the livestream from take-off to blow-up
Space X’s last animation video of the Starship’s trip to Mars
Success or failure?
For a general audience who sees NASA at work, an agency that can't afford to fail, this looks like failure. NASA failures often involve the loss of human life or billion-dollar satellites. So yeah, government explosions are bad.
But this was not that. For those who know a bit more about the launch industry and the iterative design methodology, getting the Super Heavy rocket and Starship upper stage off the launch pad was a huge success. (source)
The test flight, which for the first time launched the Starship upper stage integrated with its Superheavy booster, was, by a factor of two, the most powerful rocket launch in human history.
Lasting over three minutes, the flight test revealed numerous issues that can now be corrected in preparation for the next flight test.
Commenting on the test, Mars Society President Dr. Robert Zubrin said “The Starship test flight today was a remarkable achievement. The vehicle was able to survive numerous subsystem failures to make it through Max Q and all the way to stage separation, thereby providing a wealth of data to SpaceX engineers to now correct and then move forward.
Max Q stands for “Maximum dynamic pressure.” It’s the moment in the launch when the rocket or shuttle is undergoing “maximum mechanical stress.”
"SpaceX’s methodology is to build, fly, crash, and fix what went wrong, then try again, each time pushing further into the flight envelope. On its first try, Starship made it halfway through its flight envelope. It may take them a few more tries before they make it all the way and become fully operational, finally achieving the dream of cheap access to orbit. But they will do it. (source)
And once it can go to orbit, it will have to master refilling of its tanks there
The plan for NASA’s first Artemis moon landing, due in the second half of this decade, show the level of effort that will be necessary.
The first step in the plan is to launch a Starship configured as a refuelling station into orbit around the Earth.
A subsequent series of tanker missions then fills it up with liquid oxygen and methane. SpaceX’s agreement with nasa suggests a remarkable 14 tanker missions would be required; Mr Musk has since said it might be possible with considerably fewer.
Once the refuelling station is full, a special version of the Starship is sent up to dock with it, refuels, and heads out to an orbit close to the Moon. There it takes on board astronauts who have reached the same orbit by other means, and ferries them down to the surface. When their mission is done it ferries them back up to orbit. (The Economist)
The more we've scaled Large Language Models (LLM), the more unexpected "emergent properties" have appeared (The Economist)
The abilities that emerge are not magic—they are all represented in some form within the LLMs’ training data (or the prompts they are given) but they do not become apparent until the LLMs cross a certain, very large, threshold in their size.
At one size, an LLM does not know how to write gender-inclusive sentences in German any better than if it was doing so at random. Make the model just a little bigger, however, and all of a sudden a new ability pops out.
GPT-4 passed the American Uniform Bar Examination, designed to test the skills of lawyers before they become licensed, part of the top 10% of candidates. The slightly smaller GPT-3.5 failed it (bottom 10%)
Oxford academic Nick Bostrom on AI and sentience (conscience) (NYT)
In an interview with the New York Times, Oxford academic Nick Bostrom said that rather than viewing the concept of sentience as all-or-nothing, he thinks of it in terms of degrees
"I would be quite willing to ascribe very small amounts of degree to a wide range of systems, including animals," Bostrom, the director of Oxford's Future of Humanity Institute, told the NYT. "If you admit that it’s not an all-or-nothing thing, then it’s not so dramatic to say that some of these [AI] assistants might plausibly be candidates for having some degrees of sentience."
it’s not doing them justice to say these large language models [LLMs] are simply regurgitating text," Bostrom said. "They exhibit glimpses of creativity, insight and understanding that are quite impressive and may show the rudiments of reasoning."
LLMs "may soon develop a conception of self as persisting through time, reflect on desires, and socially interact and form relationships with humans."
"If an AI showed signs of sentience, it plausibly would have some degree of moral status," Bostrom said. "This means there would be certain ways of treating it that would be wrong, just as it would be wrong to kick a dog or for medical researchers to perform surgery on a mouse without anesthetizing it."
Crazy story from the early days at OpenAI (Wall Street Journal, unblocked)
Tensions also grew with Mr. Musk, who became frustrated with the slow progress and pushed for more control over the organization, people familiar with the matter said.
OpenAI executives ended up reviving an unusual idea that had been floated earlier in the company’s history: creating a for-profit arm, OpenAI LP, that would report to the nonprofit parent.
Reid Hoffman, a LinkedIn co-founder who advised OpenAI at the time and later served on the board, said the idea was to attract investors eager to make money from the commercial release of some OpenAI technology, accelerating OpenAI’s progress. “You want to be there first and you want to be setting the norms,” he said. “That’s part of the reason why speed is a moral and ethical thing here.”
The decision further alienated Mr. Musk, the people familiar with the matter said. He parted ways with OpenAI in February 2018.
Mr. Musk announced his departure in a company all-hands, former employees who attended the meeting said. Mr. Musk explained that he thought he had a better chance at creating artificial general intelligence through Tesla, where he had access to greater resources, they said.
A young researcher questioned whether Mr. Musk had thought through the safety implications, the former employees said. Mr. Musk grew visibly frustrated and called the intern a “jackass,” leaving employees stunned, they said. It was the last time many of them would see Mr. Musk in person.
Soon after, an OpenAI executive commissioned a “jackass” trophy for the young researcher, which was later presented to him on a pillow. “You’ve got to have a little fun,” Mr. Altman said. “This is the stuff that culture gets made out of.”
“The highest morale moment [for the business at that time] was the happy hour we hosted after parting ways with Musk,” the person said.
Mr. Musk’s departure marked a turning point. Later that year, OpenAI leaders told employees that Mr. Altman was set to lead the company. He formally became CEO and helped complete the creation of the for-profit subsidiary in early 2019.
Elon Musk Creates New Artificial Intelligence Company X .AI (Wall Street Journal and Financial Times)
As part of his AI ambitions, Mr. Musk has spent the past few months recruiting researchers with the goal of creating a rival effort to OpenAI, the artificial intelligence company that launched the viral chatbot ChatGPT in November, according to researchers familiar with the outreach.
Elon Musk recently recruited Igor Babuschkin, a scientist at artificial intelligence lab DeepMind, owned by Alphabet Inc., to helm the new effort. He has also tried to recruit employees at OpenAI to join the new lab but has had limited success, people familiar with the efforts said.
Mr. Musk’s new lab, if successful, will add yet another entrant to a heated race among tech companies to develop more powerful artificial intelligence models.
Late last month, Mr. Musk joined some tech executives and AI researchers in calling for a six month or more moratorium on developments in advanced AI technology that, proponents of the pause argue, would give the industry time to set safety standards for their design and head off potential harms.
People familiar with Musk’s thinking say he could use Twitter content as data to train its language model and tap Tesla for computing resources.
The billionaire’s potential entry to the hot generative AI market will add yet another venture to his diverse portfolio of responsibilities and investments. This includes running Twitter and Tesla, as well as founding SpaceX, his $137bn rocket maker, Neuralink, a neurotechnology researcher, and The Boring Company, a tunnelling start-up.
The new company would allow Musk to take on OpenAI, the Microsoft-backed group that he co-founded in 2015 and then left.
Since then, Musk has become increasingly vocal in his fears of broader existential threats from AI systems. He has also publicly criticised OpenAI for becoming, in his view, less transparent and too commercially minded in its pursuit of advanced AI. Musk is particularly concerned about the threat of models such as GPT-4, OpenAI’s latest release, to spew falsehoods and show political bias.
OpenAI’s CEO Says the Age of Giant AI Models Is Already Over (Wired)
Sam Altman, says further progress will not come from making models bigger. “I think we're at the end of the era where it's going to be these, like, giant, giant models,” he told an audience at an event held at MIT late last week. “We'll make them better in other ways.”
Altman’s statement suggests that GPT-4 could be the last major advance to emerge from OpenAI’s strategy of making the models bigger and feeding them more data. He did not say what kind of research strategies or techniques might take its place. In the paper describing GPT-4, OpenAI says its estimates suggest diminishing returns on scaling up model size. Altman said there are also physical limits to how many data centers the company can build and how quickly it can build them.
Although OpenAI is keeping GPT-4’s size and inner workings secret, it is likely that some of its intelligence already comes from looking beyond just scale. One possibility is that it used a method called reinforcement learning with human feedback (RLHF), which was used to enhance ChatGPT. It involves having humans judge the quality of the model’s answers to steer it towards providing responses more likely to be judged as high quality.
At MIT last week, Altman confirmed that his company is not currently developing GPT-5. “We are not, and won't for some time.”
And others big names are saying the same (The Economist)
There are good reasons, says Dr Bengio (one of the 3 deep learning godfathers), to think that this growth cannot continue indefinitely. The inputs of LLMs—data, computing power, electricity, skilled labour—cost money. Training GPT-3, for example, used enough energy to power 121 homes in America for a year), and cost Openai an estimated $4.6m. GPT-4, which is a much larger model, will have cost disproportionately more (in the realm of $100m) to train.
Since computing-power requirements scale up dramatically faster than the input data, training LLMs gets expensive faster than it gets better.
GPT-3 was trained on several sources of data, but the bulk of it comes from snapshots of the entire internet between 2016 and 2019 taken from a database called Common Crawl. In addition, GPT-4 was trained on an unknown quantity of images, probably several terabytes.
Computers will get more powerful over time, but there is no new hardware forthcoming which offers a leap in performance as large as that which came from using GPUs in the early 2010s, so training larger models will probably be increasingly expensive. Improvements are possible, including new kinds of chips such as Google’s Tensor Processing Unit, but the manufacturing of chips is no longer improving exponentially through Moore’s law and shrinking circuits.
GPUs: Graphics Processing Unit, a specialized processor originally designed to accelerate graphics rendering. GPUs can process many pieces of data simultaneously, making them useful for machine learning, video editing, and gaming applications.
Today’s LLMs, which are based on the so-called “transformer” architecture developed by Google in 2017, have a limited “context window”—akin to short-term memory. Doubling the length of the window increases the computational load fourfold. That limits how fast they can improve.
Amid the excitement Yann LeCun, one of the leading lights of modern AI (and another of the 3 deep learning godfathers), has sounded a sceptical note. In a recent debate at New York University, he argued that LLMs in their current form are “doomed” and that efforts to control their output, or prevent them making factual errors, will fail. “It’s not a popular opinion among my colleagues, but I don’t think it’s fixable,” he said. The field, he fears, has taken the wrong turn; LLMs are “an off-ramp” away from the road towards more powerful AI.
AI Agents Plan Parties and Ask Each Other Out on Dates in 16-Bit Virtual Town (source)
Using OpenAI’s viral chatbot ChatGPT and some custom code, researchers at Google and Stanford generated 25 AI characters with back stories, personalities, memories, and motivations. Then the researchers dropped these characters into a 16-bit video game town—and let them get on with their lives.
Each character also needs a memory. So, the team created a database called the “memory stream” that logs an agent’s experiences in everyday language.
When accessing the memory stream, an agent surfaces the most recent, important, and relevant memories. Events of the highest “importance” are recorded as separate memories the researchers call “reflections.”
Agent could also make plans
As the agent goes about its day—translating text prompts into actions and conversations with other characters in the game—it taps its memory stream of experiences, reflections, and plans to inform each action and conversation. Meanwhile, new experiences feed back into the stream. The process is fairly simple, but when combined with OpenAI’s large language models by way of the ChatGPT interface, the output is surprisingly complex, even emergent.
In a test, the team prompted a character, Isabella, to plan a Valentine’s Day party and another, Maria, to have a crush on a third, Klaus. Isabella went on to invite friends and customers to the party, decorate the cafe, and recruit Maria, her friend, to help. Maria mentions the party to Klaus and invites him to go with her. Five agents attend the party—but equally human—several flake or simply fail to show up.
Beyond the initial seeds—the party plan and the crush—the rest emerged of its own accord. “The social behaviors of spreading the word, decorating, asking each other out, arriving at the party, and interacting with each other at the party, were initiated by the agent architecture,” the authors wrote.
Previous newsletters:
Superhuman: see how much work a professor did in 30 minutes with AI, + more gems about the future
Tesla does something Bill Gates said wasn’t possible, AI lets you talk to younger self
That's it for this week :)
If you made it until here, well, thanks a lot for reading this newsletter! A very simple way to encourage me to continue doing this is to take a few seconds to:
transfer this to one curious friend, or share it on Whatsapp clicking here
click on the little star next to that email in your mailbox
click on the heart at the bottom of that email
Thank you so much in advance! 🙏
Here to subscribe to make sure you get the future editions if this one was forwarded to you.
More about me
I cofounded KRDS right after college back in 2008 in Paris, we now also have offices in Singapore, HK, Shanghai, Dubai and India, we're one of the largest independent digital agencies in Asia. More here.
Watch our latest game showreel: At KRDS, we take pride in designing and developing games from scratch for brands and organizations, big and small! Gamification has always been part of our DNA, since our early days creating viral apps on Facebook back in Paris as the very first Facebook marketing partner outside of the USA!
I launched 2 sister agencies:
OhMyBot.net, dedicated to designing and building chatbots (watch the video case study for a chatbot campaign we ideated and developed for Clean & Clear: The Teen Skin Expert)
The WeChat Agency for the Chinese market (the Government of Singapore Investment Corporation is a client)
I also write op-eds and do podcasts at times. Here are my latest articles and podcasts
For the French speakers:
I’ve written more than 50 articles on the future of technology over the past years, all can be found listed here.
This newsletter has a French version with slightly different content: Parlons Futur
Have a great week ahead :)
Thomas