The vibes when I joined AI in ~2008: - workshops w 50 ppl musing on whether deep learning will ever work - papers w cute toy problems - fun poster sessions - this experiment I ran in MATLAB - high-level panels on paths to AI - neuroscience guest lectures Today is *not* the same.
GPT is a new kind of computer architecture that runs on text. Yes it can talk to us, but also to much of our existing software infrastructure. First via apps on top of APIs, now inside ChatGPT via plugins. What a time right now...https://t.co/HjeUCv3XE7
Software 2.0 Neural networks are not just another classifier, they represent the beginning of a fundamental shift in how we develop software. They are Software 2.0......... The “classical stack” of Software 1.0 is what we’re all familiar with — it is written in languages such as Python, C++, etc. It consists of explicit instructions to the computer written by a programmer. By writing each line of code, the programmer identifies a specific point in program space with some desirable behavior. .......... In contrast, Software 2.0 is written in much more abstract, human unfriendly language, such as the weights of a neural network. No human is involved in writing this code because there are a lot of weights (typical networks might have millions), and coding directly in weights is kind of hard (I tried). .........
Software (1.0) is eating the world, and now AI (Software 2.0) is eating software.
.
My parents were visiting me once and as I was leaving for work I saw my mom sitting on the couch in the living room just looking forward. I’m like “mom what are you doing?”, “sitting”, she shrugged. Like not reading, listening, planning, or even meditating. Mind blown.
1 hour and 5 diagrams later I optimized 100 lines of code that ran in 13 seconds to 20 lines of heavily vectorized code that runs in 0.02 seconds, and this might just be the best day of my life, so far.
While playing around with hooking up GPT-4 to the Internet, I asked it about myself… and had an absolute WTF moment before realizing that I wrote a very special secret message to Bing when Sydney came out and then forgot all about it. Indirect prompt injection is gonna be WILD pic.twitter.com/5Rh1RdMdcV
Blueberries have joined green beans in this year’s Dirty Dozen list Blueberries, beloved by nutritionists for their anti-inflammatory properties, have joined fiber-rich green beans in this year’s Dirty Dozen of nonorganic produce with the most pesticides ....... 251 different pesticides. ...... strawberries and spinach continued to hold the top two spots ........ followed by three greens — kale, collard and mustard. ........ next were peaches, pears, nectarines, apples, grapes, bell and hot peppers, and cherries ......... A total of 210 pesticides were found on the 12 foods ........... Kale, collard and mustard greens contained the largest number of different pesticides — 103 types — followed by hot and bell peppers at 101. ......... traces of pesticides long since banned by the Environmental Protection Agency. .........
Clean 15
........... Nearly 65% of the foods on the list had no detectable levels of pesticide. ....... Avocados .... sweet corn in second place. Pineapple, onions and papaya, frozen sweet peas, asparagus, honeydew melon, kiwi, cabbage, mushrooms, mangoes, sweet potatoes, watermelon, and carrots .......... Being exposed to a variety of foods without pesticides is especially important during pregnancy and throughout childhood .......... “Exposure in childhood has been linked to attention and learning problems, as well as cancer.” ........ If exposed over an extended time to smaller amounts, people may “feel tired or weak, irritable, depressed, or forgetful.” ........ avoid most pesticides by choosing to eat organic versions of the most contaminated crops. ......... While organic foods are not more nutritious, the majority have little to no pesticide residue ........ “If a person switches to an organic diet, the levels of pesticides in their urine rapidly decrease” ........ If organic isn’t available or too pricey, “I would definitely recommend peeling and washing thoroughly with water” .
The Unpredictable Abilities Emerging From Large AI Models Large language models like ChatGPT are now big enough that they’ve started to display startling, unpredictable behaviors....... “Despite trying to expect surprises, I’m surprised at the things these models can do,” said Ethan Dyer, a computer scientist at Google Research who helped organize the test. ........ these models supposedly have one directive: to accept a string of text as input and predict what comes next, over and over, based purely on statistics .......... Computer scientists anticipated that scaling up would boost performance on known tasks, but they didn’t expect the models to suddenly handle so many new, unpredictable ones. .......... LLMs can produce hundreds of “emergent” abilities — tasks that big models can complete that smaller models can’t, many of which seem to have little to do with analyzing text. ............. multiplication to generating executable computer code to, apparently, decoding movies based on emojis. .......... for some tasks and some models, there’s a threshold of complexity beyond which the functionality of the model skyrockets. (They also suggest a dark flip side: As they increase in complexity, some models reveal new biases and inaccuracies in their responses.) ............... dozens of emergent behaviors ........... Biologists, physicists, ecologists and other scientists use the term “emergent” to describe self-organizing, collective behaviors that appear when a large collection of things acts as one. Combinations of lifeless atoms give rise to living cells; water molecules create waves; murmurations of starlings swoop through the sky in changing but identifiable patterns; cells make muscles move and hearts beat. Critically, emergent abilities show up in systems that involve lots of individual parts. But researchers have only recently been able to document these abilities in LLMs as those models have grown to enormous sizes. ................ Language models have been around for decades ............ transformers can process big bodies of text in parallel. .......... Transformers enabled a rapid scaling up of the complexity of language models by increasing the number of parameters in the model, as well as other factors. ........ models improve in accuracy and ability as they scale up. .......... With the advent of models like GPT-3, which has 175 billion parameters — or Google’s PaLM, which can be scaled up to 540 billion — users began describing more and more emergent behaviors. ......... One DeepMind engineer even reported being able to convince ChatGPT that it was a Linux terminal and getting it to run some simple mathematical code to compute the first 10 prime numbers. Remarkably, it could finish the task faster than the same code running on a real Linux machine. ................
Many of these emergent behaviors illustrate “zero-shot” or “few-shot” learning, which describes an LLM’s ability to solve problems it has never — or rarely — seen before.
............. Showing that GPT-3 could solve problems without any explicit training data in a zero-shot setting, he said, “led me to drop what I was doing and get more involved.” .............. difficult and diverse tasks to chart the outer limits of what an LLM could do. This effort was called the Beyond the Imitation Game Benchmark (BIG-bench) project, riffing on the name of Alan Turing’s “imitation game,” a test for whether a computer could respond to questions in a convincingly human way. (This would later become known as the Turing test.) The group was especially interested in examples where LLMs suddenly attained new abilities that had been completely absent before. ............... these sharp transitions ........ for about 5% of the tasks, the researchers found what they called “breakthroughs” — rapid, dramatic jumps in performance at some threshold scale. That threshold varied based on the task and model. ........... Some unexpected abilities could be coaxed out of smaller models with fewer parameters — or trained on smaller data sets — if the data was of sufficiently high quality. ......... how a query was worded influenced the accuracy of the model’s response .......... a model prompted to explain itself (a capacity called chain-of-thought reasoning) could correctly solve a math word problem, while the same model without that prompt could not. ............. using chain-of-thought prompts could elicit emergent behaviors not identified in the BIG-bench study ......... larger models truly do gain new abilities spontaneously. .......... Large LLMs may simply be learning heuristics that are out of reach for those with fewer parameters or lower-quality data........... how LLMs work at all. “Since we don’t know how they work under the hood, we can’t say which of those things is happening.” .......... They are notorious liars. “We’re increasingly relying on these models to do basic work,” Ganguli said, “but I do not just trust these. I check their work.” ........... Emergence leads to unpredictability, and unpredictability — which seems to increase with scaling — makes it difficult for researchers to anticipate the consequences of widespread use. ............... social bias emerges with enormous numbers of parameters. “Larger models abruptly become more biased.” ................. When the researchers simply told the model not to rely on stereotypes or social biases — literally by typing in those instructions — the model was less biased in its predictions and responses. .......... a new “moral self-correction” mode, in which the user prompts the program to be helpful, honest and harmless. .
Move Over, Metaverse. Here’s Something Meaner.Who’s really in charge of our online behavior? No one, David Auerbach argues in “Meganets.” ......... “Just one word. Are you listening?” Mr. Maguire said to Benjamin Braddock in “The Graduate” (1967). “Plastics.” ........ Twenty-five years later a puckish French horn player warned me, a literature major who didn’t yet have an email address, that the future lay in something called “hyperlinks.” .............. his definition of “meganet” is in essence a big blob of mortal and computing power, a “human-machine behemoth” controlled by no one ............... If the internet is the fictional doctor and scientist Bruce Banner, furtive and a little troubled but basically benign, meganets are Incredible Hulks, snarling and uncontainable. ........... “That world may not be ‘The Matrix,’ but all the connecting tissue is already there.” ........ “Meganets” made me feel deeply queasy about the amount of time I spend on Instagram, Reddit, TikTok and Twitter. Not Facebook, never Facebook — “a fount of misinformation,” as Auerbach calls it, “a petri dish in which false facts and crazy theories grow, mutate and metastasize” — except for the burner account I use occasionally to see what exes are up to. ............. a middle-aged mermaid thrashing about in the great online ocean as data floated around me, multiplying like plankton........... “Reality bites,” we naïvely thought, but here “reality forks,” with blockchain doubling back on itself like a caterpillar. “No Rousseau-esque ‘General Will’ emerges from the bugs and forks,” is the takeaway............ Aadhaar, India’s national identification program: “a unified, government-sanctioned meganet” ........ a virtual pandemic called Corrupted Blood that spread through the video game World of Warcraft in 2005, arguing that “the distance between Corrupted Blood and a global financial meltdown is smaller than you think” ............. “We search for where the power really lies, when it does not lie anywhere — or else it lies everywhere at once, which is no more helpful.” .......... “If Big Brother can’t be stopped, we should focus on throwing sand in his eyes rather than futilely trying to kill him.” ........ Take my Wi-Fi — please! .
Meet the Editor Behind a Slew of Best Sellers Jennifer Hershey is the guiding hand who helped shape “Daisy Jones & the Six,” “Mad Honey” and many other chart-topping regulars. ....... how much more nuanced and honest this book is because of you.” ........ She’s the publisher and editor in chief of Ballantine Books ....... “Sometimes we gather as a whole team — the publicity person, the marketing person, the publisher, the editor, all the people who worked on the book — and we call the author together. There’s so much joy in that moment, and definitely a lot of tears. It’s not even so much the hitting the list but what it symbolizes: that an author’s work is reaching people, that their voice is being heard and that readers out in the world are connecting to their words.” .
Netherlands and Japan Said to Join U.S. in Curbing Chip Technology Sent to China A new agreement is expected to expand the reach of U.S. technology restrictions on China issued last year. ........ sweeping restrictions issued unilaterally by the Biden administration in October on the kinds of semiconductor technology that can be shared with China. .
